that was really cool. thanks for sharing i suppose you can do some of that.
you can use a ground array to display the animations on. but i dont know how you can do those animations. i guess they need to be pre-recorded video files.
and instead of using a pen, you can use a toolbar marker type marker (something small).
so let say you have a pre-recorded walking man, and a stable man figure. your program will see that there is a gap in the ground, so it will show the man walking and moving and stopping at the gap at that point it will show a stable figure. then using your
small marker, you draw a line between the gap, and your program knows the gap is filled, then you play the pre-recorded video and the man continues to walk on your ground array.
something like that should work i guess.