I have not managed to break the multiple face tracking and replicate the bug in your video. This is something that would require further investigating. I would like to know if you could replicate this bug repeatedly. It could be either hardware or software that is causing the bug. For one, the camera only allow one application to read the web camera stream data from what I have tried on Windows. Since the web application does not write any log files onto your computer or communicate to a server, it will be harder to pinpoint the cause of the bug.

Can you open up the web inspector ( should be F12 shortcut )? See if there are any errors in the console section.

Misaki wrote

One thing I noticed is that the model that was initially tracking my movement will follow the second face’s movement when the second face tracking starts. Is this something wrong with my settings?

It is not your settings. The face tracking system is probably scanning the camera image from the top to down and left to right direction. The second face that appeared after is identified as face 1 because it is top left of the first face.I am going to add an numeral indicator near face meshes later to help identify the face index.

I think I have located the problem from the error message and fixed it. Try again and see if you still get the error. For some reason your web browser was generating incomplete data. I made the program check if the data is missing beforehand so it would not stop working :wounded: .

The good news is that the previous fix worked but the same problem occurred on another section. I did the same check up on that section as well. I hope I squashed all those bugs :wounded: .

I am surprised that those errors has not appeared on my computer with multiple face tracking :think: .

Now it's stable and working! Awesome

Great to hear.

However, as you can see in the video, only the model placed on the right side sometimes momentarily appears to be in a strange look where each facial part seems to pop outward.

I had the animation track scaling factor too high for when the AI stopped recognizing a face. The initial value of the vector has been adjusted by a million fold. Hopefully that will not produce an image that is as jarring as your image. I added another circumvention to stop moving the Spine model in that situation as well.

I don't know why the tracking is off, especially since the face is not moving significantly, but the face closer to the camera appears stable, so it may not be stable if it is farther away.

I lowered the face recognition threshold from 80% to 50%. When the AI is at least 50% confident, the computer will start producing facial coordinates for the Spine model. Hopefully that will help with finding faces further away from the camera. I am not sure if I want to expose the ability to change the threshold in the future.

I think the eye issue could be resolved by editing the animation in Spine.

Anyway, it is wonderful that this tool has improved very much in this short period of time :yes:

Good thing they were simple fixes :wounded:.

- It would be cool to see what files are added to the drag and drop zone when they're dropped in to know that they've successfully been uploaded without pressing the button below to see if it works.

Yes I can do something about that.

- I'm not sure what happened, but when I imported my latest rig, the import wouldn't load all of my assets. It worked when I used an older file though, so it may be an issue with my file and not the prototype (the same thing happened recently when I used that file with Rhubarb as well). But I thought I'd at least mention it (see bad import jpg)

I do not know either about the bad import. Would be hard to figure without any of the files or error log.

- Being able to adjust the strength of individual parameters was really helpful here. Before I recorded, my eyelids wouldn't open to their full open position so I adjusted that a bit

Yeah, not everyone is comfortable or able to stretch their facial features to the extreme. Even if they could, their face would fatigue eventually.

- I made a rough test of some eye blinks which I show in the video. I experimented with turning alpha to 0 on the lower lids so they would disappear when the eyes opened. It would be cool if there was a way to disappear the lids just as they open all the way.

• It would be even cooler if there was a way to use an entire animation as the motion rather than a single key (unless I set mine up incorrectly). For example, having one animation control the entire left eye blink. The first frame would be eye completely closed, and the last frame would be the eye completely open. Then the frames in between could be refined to allow custom deformation rather than a linear straight shot from one position to the next. This could also help in selecting the best time to adjust draw order too. I'm thinking of the Moho rigging process as inspiration here.

The problem is that the live timeline ( from face tracking ) would conflict with the animation timeline using animation mix alpha setup I have now. I think it is still possible but it would require the calculations to be applied to the animation track timeline itself and have the animation mix alpha constant at one. This would require me to create a separate copy of the application to experiment with. Wait for the next update. :nerd:

I tested with only with one eye but your request of able to use more than one keyframe for Spine Vtuber Prototype is possible! I keyframed at half way mark on the timeline with green eye and the full way with red eye. I had set the FPS to 100 so each frame is 1% of the movement range. You can set the FPS to any value but you have to keyframe within the one second. I still need to make the changes for the rest of the animation tracks and update it on itch.io :cooldoge:.

Spine Vtuber Prototype 1.0.5 update

• Moved the names of uploaded files from bottom of the page into the Drag and Drop Zone area. This allows easier viewing of loaded files.
• Changed how calculations are applied. This allows animators to use multiple keyframes on the timeline for each animation track. Animating within one second is recommended. Previously, each track only allowed one keframe on the timeline.

https://silverstraw.itch.io/spine-vtuber-prototype

I have also updated the Spine Vtube Test Model to reflect the change in 1.0.5

https://silverstraw.itch.io/spine-vtube-test-model

I was able to recreate your error and patched it. I forgot that drag and drop uses a slightly different code than clicking to upload files.

The weird state of your model is normal and is separate from the error. Before you were only allowed to keyframe the end state of the animation movement. Now that you can key more than one frame, you are going to have move the previous keyframe to the where 1 second would be at. For example. if your Spine file is set at 60 FPS, 1 second is on the 60th frame. If it set to 30 FPS, then 30th frame is the 1 second mark. Then you are going to need to add a keyframe at the 0th frame. It is going to take some time to make the changes to the Spine file but you are free to keyframe whatever you want within that 1 second timeline. Note: I set my FPS to 100 so that each frame is 1% of the movement range.

Excellent. You got your model to work now.

I also noticed that your model animation got smoother from the video. I think the update definitely played a role. As to why, I would not be able to provide a concrete answer. I speculate it could better in-betweening when there is more than one keyframe. The other thing could be it better performance changing the track time as opposed to changing animation track alpha. Nate would probably know more about it.

The " the mouth would open wider than specified in the 'mouth height' animation" is a number capping issue. Out of the box, Spine runtime caps off the track time with track ending time while the animation track alpha is uncapped.