Very cool! We have plans to do this officially. It's still early days, but it's super cool!

You did! :punch:

Besides the camera/vtubing itself, we also have some supporting features in the works that makes vtubing super cool: physics! We're in a bit of a rut though, as we started these things and then switched gears to improve the new graph. Lately there's been a lot of distractions that eat our time and/or fracture our focus, plus the holidays, so even that's going much more slowly than we want. We do keep plodding along though, so we'll get there in time!

Oooh now that looks very promising as well, SilverStraw!
I'm so excited about being able to use Spine rigs for streams >.<

Super cool! From the debug render on the left, it seems we are using the same tensorflow light model

Cool! It can also be done via drag and drop. Working with browsers is such pain!

Neat!

Great, I’m so excited!

OMG, it looks very easy to use! The mouth behavior looks natural Although the character has no eyelid yet, just being able to open and close the mouth conveys face expression quite well.

Great evolving! :yes:

Congrats on the release!
I tried with a simple character which was created based on your Spine project file and it looks good! :heart:
Here's what I tried:

Sorry about the lack of blink animations since I drew and rigged it in a hurry and about the closed mouth not being clean enough! I will make various modifications later.

It may depend on the environment to do this, but it seems that eye pupil movement tends to be a bit jumpy. However, the response for facial tilt, direction, and mouth movements are very smooth! Also, the setup was easy to understand, so creating my own project was easy for me. It is a lot of fun to try out the movements with this tool

I have added some modifications to my skeleton:

Eye glasses affect the reliability of eye pupil tracking.I think it's because of the glare on the eye glasses. I used another software called Vtube Studio and it also has this issue. Eye glasses are a bane to eye pupil tracking.

Unfortunately, I tried it without eye glasses last time and this time, but it seems the eye pupil tracking response is not very good :tear:

Currently, the right eye and the left eye can register separate animations, but since it often happens that only one eye moves and looks weird, it would be good to make both eyes share the same animation once. Or, as in Owl's sample, it would be good to combine the animations for the face and eye direction together for now.

I have OBS installed, but even if I started the virtual camera, I could not use it because the option does not appear in Google Chrome's camera settings. To be precise, in the general settings of Chrome, I can set like the following:

However, on the actual web page, it was automatically changed to FaceTime HD Camera and could not be freely changed here.

Incidentally, Google Meet allowed me to choose the virtual camera.
I have looked for other ways to flip the camera, but unfortunately have not found any.
I have attached the project file above so you can see if the problem does not occur in your environment.

Thank you for your quick fixing, it looks much better! The movement has become smooth, and weird movements such as only one eye moving are less than before.

In the video above, I tried to roll my eyes 360 degrees, but unfortunately the up and down did not seem to respond very well. The distance between up and down is shorter than left and right, so it seems to be hard to catch the movement. I'm not sure if it would be optimal to have the filter adjusted based on my model, so I think it would be nice if the settings could be adjusted by the user.

By the way, I will modify my model later as the rigging of the eyes and mouth is still not very good. I'll share it again when the improvements are done.

So the jitter filter is for the model to stop twitching so much. Much like the .gif below when your eyes are not moving but the model is twitching excessively.

Ah, I see.

When you allow those adjustments, users would want a way to save the settings so they do not have to adjust every setting each time.

Exactly. I think it would be great if you can download the settings as a file such as a JSON format and load it with the skeleton data when you want to apply the settings again.

There are so many features to add, so I am going to implement them one at a time.

Sure, take your time

Misaki, have you played with the drop down box below the Reset Render Camera button on the Model Settings? I might have not made it clear that drop down is where you change certain properties about the model(s).

I have tried some of them! The position and scale of the model can be changed on the canvas with the mouse (this is really comfortable!), so I have not had to change them using the settings box, though.

While over the canvas, your mouse only changes the viewing port. The model's scaling is still at 1 scale for x and y.

Ah, I see. I'm looking forward to seeing what fun looks like when multiple models are placed.

Regarding animation property, what is the middle box set for? I can see that changing the animation of the pull down changes the animation, but is it played on track 0?

Regarding skin property, I can confirm that it works correctly, but is it currently only possible to apply one skin?

Thank you for elaborating on those!

If value input box is too confusing, I could hide it for properties that do not need it.

Yes, I think the value input box should be hidden as it would mislead people into thinking it is something that is meant to set up the animation.

I forgot to update the track layer for the animation property so it is not working on the intended track layer. It is in the middle of the track stack so you have half the tracks overriding it.

Ah, that makes sense. I had a feeling that something was wrong. It would be useful if it could be used to switch from the default breathing animation to another, or to switch facial expressions.

So far you can only apply a single skin. I only worked out populating the drop-down list of all skins. I never had the intent of multiple skins in early development but it seems like something I could expand upon.

Yes, it would be useful to be able to use more than one skin, such as one for costumes and one for expressions or postures, since skins can be used to change facial expressions and postures using constraints.

I give you my suggestions on many things, but of course this is your tool, so make it what you want it to be

I was hoping to soft cap the amount of animation tracks and skin layers. If I follow your suggestion, everything would need to be uncapped ( or until you web browser crashes :p ). So far I am using 20 something animation tracks just for face tracking.

I'm not familiar with the cost of having a lot of tracks, so it would be better to have Nate or Mario comment on this.

On a side note, this topic got so many views within a day. I checked my itch.io stats and there hasn't been that many downloads on the test model. So many lurkers among us or it is just you, Misaki. :o

I’m sure I'm not the only one checking this topic! The order of topics will go up when there are responses in this forum, and a lot of people see the posts at the top. That’s why there are a lot of views.

It has been a while, but I have made various minor modifications to my skeleton. The latest version of the recording is here:

As for the old videos, I had set them to limited public access, but I have set the latest video to public access. For people who are new to this tool, I have recorded the process from uploading the skeleton data so that it can be a simplified tutorial.

My skeleton still has some issues as the clipping attachments for the eyes sometimes go wrong, but I have attached the latest file here for your reference:
face-for-Spine-Vtuber-Prototype.zip
(Also, I deleted old files in this thread.)

I will come back to this thread when I have time. Cheers! :beer:

That sounds like a great update! However, I could not find where the strength setting was. I don't see the drop down menu in your video, is it visible on your end?

Hello. What is wrong with the clipping attachments for the eyes?

When the eyes were closed, the clipping masks sometimes crossed over, and this caused the eyes that should have been hidden to be visible. I fixed the problem today and I replaced the attached file on my previous reply.

Hmm, somehow I can't find that setting on my end....

Also, I can't find "maximum number faces", so the update seems not to be reflected properly. Is there anything I need to do to use the updated version?