Sprite Shaders for Unity

ToddRivers

Yeah I'm trying to use the same shaders for as much as my project as I can (which is always a good thing to aim for). There's some scenery with actual 3d geometry I'm using my sprite shaders for and so for those I still use the mesh normals but like Pharan says for Spine sprites it doesn't make sense - they're always gonna be the same / flat.

Ok I've added on last shader feature (for now!) - the option to tile or offset textures. Probs not needed by most peops, I'm using it to scroll textures so thought I'd share anyways!

Pharan

Why is the fixed normal a float4/Vector4 instead of a float3? (more things I've seen for the first time)

ToddRivers

Oh its only because unity shaders only allow exposing vector4s. In the latest shaders it appears as a Vector3 in editor though, plus the actual shader code never uses the w value.

Pharan

Really? That's super weird.

ToddRivers

Hey! Sooo I've realised there's a pretty big problem with normal mapped lighting!
Dunno how I missed it before - The tangents aren't always fixed in sprites like I first stupidly assumed - the tangent should point along the horizontal axis of the texture for a vert. For most meshes in a Spine animation this should be along the x axis so setting the tangent to (1,0,0,1) made sense.

BUT there are times when this is not the case, especially any time mesh deformation is used and verts get rotated etc. Or when mesh attachments have rotation. OR anytime the skeleton in flipped.
There's no getting around needing to calculated the tangents properly based on the uvs basically.

I've attached a tangent solver I nabbed from here http://answers.unity3d.com/questions/7789/calculating-tangents-vector4.html which seems to work nicely.

But you'll also need to change the SkeletonRenderer to make use of it, I modified the normal calculating code chunk starting at line 577 to use the attached tangent solver, plus I've moved the rebuilding of tangents outside of the calculate normals check so you can just build the tangents if you want to make use of fixed normals:

if (calculateNormals) {      

   Vector3[] normals = new Vector3[vertexCount];
   Vector3 normal = new Vector3(0, 0, -1);
   for (int i = 0; i < vertexCount; i++)
      normals[i] = normal;
   (useMesh1 ? mesh2 : mesh1).vertices = vertices; // Set other mesh vertices.
   mesh1.normals = normals;
   mesh2.normals = normals;
}

if (calculateTangents)
{
   SpineTangentSolver.Solve(mesh1);
   mesh2.tangents = mesh1.tangents;
}

Stevepunk

I've tried this shader with the pixel lit option and the Spine Dragon.
Unfortunately some of the parts appear to be lit from the wrong angle.

First the normals as generated by the latest version of Sprite Dlight:

Next the results with a directional light pointing to the left:

Some of the parts seem to be lit from the left or from below instead of from the right..
I've downloaded the c# script which presumably fixes this problem though I'm not sure where to locate it in my project..

ToddRivers

Yeah that looks like it could be the tangents, plonk that SpineTangentSolver.cs file anywhere you want inside your Unity Assets folder.
I'd search for 'SkeletonRenderer.cs' and put it next to it. You also need to make the changes to SkeletonRenderer.cs, I've attached mine with the needed changes in, so just replace SkeletonRenderer.cs in your project with this one.

Hopefully the tangents solver or an equivalent one will be inlcuded with Spine in the future 🙂

Pharan

@[obrisani korisnik]
I think you're running into what they were trying to solve with Project Magelight.
It would've been great if this could be done in the shader. It's too bad vertices are oblivious to one another in a Cg program.

How do tangents figure into these lit shaders anyway? Do they just point to the right? Why would that break for FFD/weighted meshes and flipping?

[EDIT]
Oh, I see. Normals are either stored in object space, or in tangent space. Another one of those must-match-however-you-saved-it scenarios.

I still don't get why that means it will break. Seems like it would be a reason for it not to break.

ToddRivers

hmm, both the normals and the tangents are in model space. Normals will always be the same for Spine animations but tangents are unknown & have to be calcluated. They are only used for bump mapping though.

So for bump mapping the shader ultimately needs to work out a normal direction in worldspace to bounce light off correctly. This is done by reading a direction from the normal map texture, and then converting it into world space.
To do this for any vert you need the model space normal (which yep will be (0,0,-1) for spine sprites), plus the modelspace tangent which can be any perpendicular direction to that normal. The tangent will let the vert know what way 'up' its normal map textures is by pointing along the direction of its mapped texture.

If you had a spine animation where every textured mesh in it had its texture aligned with the screen (ie not rotated / just dragged and dropped from the image folder) then all the tangents would be (1,0,0).
If one of those meshes was rotated by 90cw then its verts tangents should be (0,1,0). If the tangent is still (1,0,0) then the world space normal caluclated by the shader will be incorrect (ie also 90 degrees out), so you get some meshes lit incorrectly like the dragon above.

So for correct bump mapped lighting in Spine we dont need to pass the normals to the shader in or even store them on the mesh, but we do need to calculate tangents using something like that tangent sovler. They might be a more optimised way of doing it though!

Pharan

I get it now.
I guess one option is to store it with the attachments at load time, and have their ComputeWorldVertices method return bone-transformed world vertex tangents accordingly for SkeletonRenderer to use.
It's too bad those are on the spine-csharp side but that shouldn't be a hindrance for people who are really after it.

Stevepunk

I've replaced SkeletonRenderer.cs and included SpineTangentSolver.cs but it doesn't seem to change anything..
Huge thanks to you Todd. You've done a lot to help and you're not even part of the spine team :clap: , but this really shouldn't be necessary.
3rd party tools shouldn't be breaking core parts of unity..

The problem seems to be related to the way that spine rotates the images (maybe in local space instead of tangent space??), as non-spine images don't have this problem when rotated.

I can rotate the sprite map and the angle is known by the unity lighting engine and the lighting on the surface is updated correctly.

However for spine models using the spine system (even the ones that are then built in unity), any rotation on objects is done in a way that the unity lighting system doesn't recognise.
Even if I manually rotate spine models in unity (even ones built in unity), the lighting doesn't update to respect the rotation..

However if I use unity's standard bone animation I can use standard or custom shaders and unity's lighting respects the rotation of the object.

If this is not fixed I'll probably just give up and go back to the standard unity animation system.

Spine did seem like the most popular system for unity animation but it completely breaks other parts of Unity.. 🙁

ToddRivers

Hmm sorry I think I might've uploaded a broken SkeletonRenderer.cs.
I do have go this all working locally so it is possible, but yeah Spines unity runtime needs to be changed to build proper tangents - without them bump lighting just can't work.
Could you try downloading the shaders package from the first post again? I've added my current skeletonrenderer and tangent solver to it. Hopefully that should sort things, its def working for me now finally!

Also I've added a 'SpineFlipper' component I've been using to flip spine animations in the editor, which you should be doing instead of rotating the sprite 180 around the Yaixs or giving it negative scale. (Just add the component to a gameobject with a SkeletonAnimation and then you can set the two flip flags to flip animation in the scene view).

@Pharan I've also found a bug with the BoneFollower component which means it doesn work properly when the skeleon is flipped. I can share my fix with you if you want?

Stevepunk

ToddRivers wrote
Hmm sorry I think I might've uploaded a broken SkeletonRenderer.cs.
I do have go this all working locally so it is possible, but yeah Spines unity runtime needs to be changed to build proper tangents - without them bump lighting just can't work.
Could you try downloading the shaders package from the first post again? I've added my current skeletonrenderer and tangent solver to it. Hopefully that should sort things, its def working for me now finally!

Also I've added a 'SpineFlipper' component I've been using to flip spine animations in the editor, which you should be doing instead of rotating the sprite 180 around the Yaixs or giving it negative scale. (Just add the component to a gameobject with a SkeletonAnimation and then you can set the two flip flags to flip animation in the scene view).

@Pharan I've also found a bug with the BoneFollower component which means it doesn work properly when the skeleon is flipped. I can share my fix with you if you want?

I've re-downloaded the attachment from the original post and started a new project as the original one seems to have coloured all of the textures a flat bright pink.

I've imported spine-unity and SpriteShaders.unitypackage.

I've baked a dragon from the Examples folder and changed the shader on the dragon's material the Game/Sprite Pixel Lit.

I've included a directional light and am getting the same results as before.

However, I haven't been able to find this window:
Loading Image

The script "Skeleton Animation" doesn't seem to exist in my project:

These are the scripts I can add as components to the prefab:

Ok I've switched to an example scene which includes the dragon as a spine object rather than a baked unity object.
This time I get the Skeleton Animation component appearing and I have selected the 2 required options.
However, the lighting is still incorrect. This is most obvious on the tail where some segments are lit from below and some from above:

Also, the rear leg is lit from above but the rear foot is lit from below.

26 Mar 2016, 11:31

Here's a video of spine lighting from over a year ago by the developer.
There seems to have been no progress in over a year on this 😢

This is the effect I'm wanting.
How do I achieve it?

26 Mar 2016, 12:48

I've exported a brand new version of SpineBoy with atlas rotation turned off.
When I light him up in Unity the lighting is now correct on all body parts!

Success!

Actually, not quite..

As soon as I select an animation, the lighting doesn't update correctly based upon the new rotations 😢

As you can see, his rear leg appears to be lit from the bottom by the bright blue/white light (really it's being lit from the top right).

But he does look really good with just a couple of lights.
It's a real shame that spine isn't compatible with unity shaders and lighting
(unity shaders are one of, if not the most important, parts of unity - without shaders nothing would be drawn to the screen - I can't comprehend how an animation modelling system wouldn't be compatible with the core process of drawing things to the screen - it kind of defeats the purpose)

Thanks Todd for all your hard work on writing custom shaders for spine, but really spine should be compatible with any of the other custom shaders for unity and most certainly the default shaders..

I could go one but there's no point.
I'll just get a refund on Spine and if they ever fix their tool to be compatible with Unity I will purchase it again.

Mitch

Normal mapping 2D things is complex. 3D lighting on 2D things is complex. None of these things are now nor have ever been part of Spine. I'm sorry you were deceived, but MageLight is a personal project of mine and a friend's and not part of Spine.

We'll release it into the public eventually.

Pharan

@ToddRivers. Sure. Let me see.

Also, since we're pulling on the same thread as Mitch was, the theoretical solution for the simple case is to rotate the tangent according to bone rotation + region offset rotation + setup pose rotation. That setup pose+offset value thing is probably where storing a value in the attachment class is helpful.
For meshes and weights, I guess the theory is still to have its CalculateWorldVertices also rotate its tangents, taking FFD and weights into consideration. The "setup pose" tangent values are probably according to UVs like you said.

Cranktrain

Having followed this thread for some weeks, I've started to experiment with the shaders:

Loading Image

The developer of Sprite Dlight very helpfully exported a couple of normal maps to test with, clearly there are some unpleasant seam artifacts that you can spot between limb sections, we're talking about ways to sort that out.

Running into the problem that Stevepunk described, with the tangents not taking into account the rotations. It sounds rather complicated to me, anyone think it's solvable?

Pharan

For region attachments, it's solvable; no doubt about it. Meshes with weights and FFD is a bit complicated.

The pillowed joint seams are a problem on the normal map generator side though. Those programs need to have a way to define areas where parts need to blend into each other.

Cranktrain

Pharan wrote
For region attachments, it's solvable; no doubt about it. Meshes with weights and FFD is a bit complicated.

That's great news. I think supporting attachments would be the minimum, and maybe for most uses it would provide results that would be 'close enough', despite slight inaccuracies when vertexs are deformed.

Pharan wrote
The pillowed joint seams are a problem on the normal map generator side though. Those programs need to have a way to define areas where parts need to blend into each other.

Yes indeed! I'm less concerned about this one.

Found an issue with the shader, I tend to use slot colouring to shade different parts of my game's zoo visitors, but the shader seems to colour them with a pinkish hue?

Loading Image

What's going on here?

ToddRivers

Hey sorry haven't checked the replies on this thread for a while!
Locally I have lighting working with normal maps perfectly now so am a little confused why its not working for you guys - let me put together an example scene based on the sprite unity example to show how I'm using everything. It should all be easy enough.
One thing to look out for is how you generate sprite normals - they need to be facing the correct directions themselves otherwise things will never work. You can in theory pack them rotated but make super sure their all generated facing the same direction. I'm using Sprite Illuminator by the TexturePackerPro guys and that seems to be pretty awesome.

@Pharan The tangents are working even for deformed meshes using my current method. Using a solver function that calculates the tangent for each vert based on the other verts it shares a triangle with and their UVs means it will always be correct. It just needs to only happen when UVs or verts actually change.

@Cranktrain huh, I guess slot coloring just changes the vert colors behind the scene? Don't know why that wouldn't work.. could be a bug with the shaders I've never used vert colors. Let me check!

Pharan

Ah. Good to know.

By the way, that's correct. Spine's colors are applied as vertex colors.

« Prethodna stranica Sljedeća stranica »