Dailies and Old Work

To keep myself motivated I'm hosting dailies over on blenderartists.  Join in if you like, criticism is always welcome!

Secondly, its always really nice when someone posts some of your work, even if its an old piece that you're not particularly proud of any more, its good to hear that people are still enjoying watching it.  Cycling blog Two on Four Wheels featured one of my pieces from last year, Au Soleil.

Things That Go Bump In The Shed

Thank god for Bullet Rigid Body Sim, saving animators' time.  Only 38 crashes and two corrupted files (thankfully I had backups) later and I've got a second and a half of falling spades to show for it (I've hidden all the characters by the way, so as not to spoil any surprises).  Sometimes I think there are demons in my computer.

Those warped meshes on the right aren't me forgetting how to model, they're just shadow catchers for the comp over the matte painted backgrounds which should be pretty funky.

Big thanks to this post on blenderunderground which reminded me how to use Bullet - its been a long time.  Looking forward to a merge of Aligorith's Bullet GSOC soon.

Edit: on second thoughts, this had my little brother and I in stitches... we're easily amused.

Freestyle Rendering

Got a shock to find this one staring back at me today.  Some wrongly connected nodes and textures giving a freaky render.

Here's a preview of the node graph I'm using to mix up the AO, Shadow, Flat Color and Freestyle Passes, otherwise known as the 'spaghetti system'.  It masks the freestyle pass using objects from the sets, edge detects to give a line round the shadows to match the Freestyle line, and adds a slight transparency to the characters when comping them over the backgrounds.  The backgrounds get a slight blurred underneath and just around where the characters are to help the characters stand out a bit.  I'm also mixing in ink and paper noise as freestyle's textured stroke rendering isn't supported yet in blender as far as I can tell.

I'm working with three Freestyle shaders (and a common functions module) which I coded based heavily on T.K.'s fantastic examples in the blenderartists thread.

I'll post the source when I've got them finalised.

Back to animating.

Spin me Right Round, Like a Record

Not 100% there with the look so far but it will probably do for the national gallery version.  I should have just animated the whole film like this with their feet sliding along the floor like this and everyone in their default T pose.  It would have saved time on the rigging and the animation would probably be just as good as it is now!

Cloth Sim for the Farmer's Coat

So manually animating the coat didn't go so well.  Instead I've created a low resolution version of the lower half of the coat mesh,  put the armature, then the sim on that, then a solidify modifier on top of that.  To connect the high resolution coat to the low resolution sim mesh I use the normal armature deform for the top half of the coat (arms etc), then use the low resolution mesh with solidify modifier as a mesh deform cage for the lower half of the coat.  You need to do all sorts of weight painting and multi-modifier tricks to get the two to play nicely together, then you need to have a doubled up copy of the armature for the coat with copy transforms otherwise you'll run into a dependency cycle (this depends on your rig of course).  If you're going to use cloth simmed objects linked into files as groups with proxied armatures you need to then link in the cloth sim object and the collision objects again after you've done your usual group linking.  Overly complex I know, but it works... just.

Rigging Done

He's multi coloured in proxy form to help me spot mesh intersections.  The coat was an absolute nightmare to rig but now I have a fairly nice fake wind effect using displacement textures and animated textures, all controllable at animation time.  Time to start animating.

Say What Again: Facial Rigging

I primarily designed this rig around subtlety of expression and emotional nuances, obviously.  First full 'stop staring' style rig I've built, quite happy with it so far.  I rigged his mop-top as well so I can push it out of the way when you need to see the brows.  Did the crow and the dogs facial rigs as well  and completed the bike rig yesterday (with moving chain) as well as working up a fake wind sim system using displacement modifiers.  Just the farmer to finish off with then animation with UV unwrapping on the side.

Click What? Wing Rigging Revisited

So a few weeks back I demoed a folding wing ring based on Jared Reisweber's notes.  It didn't look much like a crow though - it was more of a soaring wing type.  I spent most of today pushing verts around and working with shape keys to get the fold even better and solve the distortion problems at the extreme poses.  I'm using a solidify modifier post-armature on the feathers to take give them some thickness - I had to separate them out from the rest of the mesh after binding. Most of the above bones will be hidden at animation time leaving 4 main controls per wing and 8 tweaks for follow through and puffing out etc.  Not sure what I'm going to run with for simming yet - maybe just animated displacements.

Jonny's Fixie

Yes, I wish it was real too. Freestyle render in Blender 2.5. Not rigged yet.

Dog Topology

Here's how 'Blake' the border collie sheepdog is looking in wire form so far - 2100 faces, all quads, nice clean mesh topology with maybe just a few too many poles.  Struggled a lot with the head.  Wish I'd had access to a real dog to help with the modelling of this.  I've only given him front teeth as he rarely opens his mouth in the 'Crows' short.

Making good progress on the rigs.  The biped one is ready for re-targetting - a hybrid of my 'eskimo' and 'bally' rigs - using iTaSC for damped IK - no more popping!  The quadruped rig just needs layer organising and testing.  Jonny is pretty much ready to bind, just finishing off the Farmer's haircut, hat, glasses and coat.

Today's Top Tip: Workaround for buggy auto-weighting in Blender

If you're using auto-weighting in blender and you come across a complex mesh where it just isn't working - you probably have flipped normals.  Just tab into edit mode and W>flip.  If that doesn't work  my workaround was to build a much lower resolution mesh with nice clean topology for auto-weighting, then use this script which I quickly cobbled together to proximity copy weights.  It will copy all the weights from the source object to the target object, but be warned it will overwrite any existing vertex groups on the target, which you might be using for other non-binding related tasks.

Fake Plastic T-Pose

Jonny's nearly done, just some more work on the hair and cloth collision meshes before texturing, shape building and rigging.

T is for Topology

...with which I'm wrestling....

Jonny's hands, arms, t-shirt, jeans and pumps meshes all work-in-progress, the most complete of my models so far, with some subtle asymmetry.  The head meshes and the farmer and dog are also pretty far down the line but still messy shape-wise.  Its awesome to have access to the tube project's character meshes as a topology reference, one of the perks of being an intern on the project!  Working in blender-2.57-freestyle - hopefully will have some python shader code going soon!

Task Project

The task project has been the forgotten orphan for the last few months while I've sneakily been preparing my storyboards for my graduation film way ahead of college schedule.  Safe to say my graduation film will be a lot stronger as a result of this diversion, but on the other hand the last three days have been a whirlwind of hectic animation as I've caught up with the 8 weeks work in a vastly condensed timescale!  Anyway here's my task project, 'Rejection', a largely unrealised idea.

Demo Video

Here's the (almost) completed addon, and here's a demo video showing a real test case.  Note how I'm working in the 3D view the whole time and in the background the script is adjusting all the F-curves while I work.  Obviously you don't need to have the graph editor open, but its nice to remember what animating used to be like!

Quaternion Beziers in the 3D View

Quaternions might have great benefits for key-framing rotation (no gimbal lock and so on) , but I have to admit that when it comes to the 2D graph editor I don't stand a change of working out what effect of grabbing the handle of a key on a Quat's W channel will have on my animation.  Generally I'll adjust the key in the 3D view and leave the bezier handles well alone set to auto (or vector if I want a snap) because independently changing any of the W, X, Y, or Z channels is unlikely to yield any great improvements for my animation!

Today I expanded my add-on to draw the handles for Quaternion and Euler rotations, and to show the path of the tail of the bone (which is probably what we want to track when we're adjusting rotations).  The timeslide tool still works but I need to do a little more coding (and get my head around what a normalized quat is) before grabbing the bezier handles in the 3D view is possible.

EDIT: http://www.pasteall.org/19807/python - now with quaternion and euler editing!  Editing preview shown below.  At 1500 lines it outweighs the mushroomer, the crowd sim and the autowalker as my new biggest project... not bad for 3 days coding!

F-curves in the 3D View continued...

Today's progress... 3D Bezier F-curves are now editable in the viewport for unconstrained child bones with inherited rotation and asynchronous/independent curves!  I learnt a lot from studying Crouch's script which has a larger but different feature set and currently works with objects not bones (mine currently works with bones and not objects).  Hopefully my script shown below, will help push the discussion along and go some way towards bringing this feature to blender in a finished state before it hits maya!



The script is available here but be warned its in heavy beta, so back up your animation first!  Feel free to bug report in the comments below.  I've coded an on screen help (which will start when you enter the 3D f-curve edit mode) as the shortcuts had to be unusual to save messing with the keymaps.

UPDATED: http://www.pasteall.org/19807/python

There's extensive discussion of the autodesk technology preview which sparked all of this here (though its not really a new technology at all as apparently DigitalFish Reflex had it a few years back).

In other news an animator whose work I've been following recently is Dustin Grella.  If your mind is buzzing as much as mine with all that discussion on 3D fcurves you'll probably find it refreshingly low tech... here's Bag for a Banana Peel.

Editing F-curves in the 3D View

So after feeling really proud of myself for getting asynchronous f-curves represented in the 3D view in blender with live updating, I read of Crouch's concurrent efforts to bring the same functionality to blender and realised I'd been beaten to it!  Crouch's script only works with objects and has a very cool visualisation system for speed and acceleration as well as the 'time beads' functionality that maya's technology preview provides.  It doesn't represent asynchronous curves in the same way I've done below, but then maybe I'm going over the top.

Tonight I managed to get the script working with child bones - it has to cache the parent space in advance.  The great thing about open source is that I can borrow and learn from crouch's code.  My work in progress code is linked here but be warned its still quite messy and for interest's sake only!  UPDATE: http://www.pasteall.org/19807/python

The Computer Drawing Experiment...

Since implementing the 'Scribbler' for my earlier 'spiders' demo video I've started to think more and more about how I draw.  I wanted to see if I'd be able to write an algorithm which would draw as efficiently as humans do.  Firstly I adapted the scribbler to only draw with one long line and combined this with my optical flow distortion code, again using OpenCV for optical flow analysis and Blender as my drawing interface.  The scribbler would need to know where it had been recently so it didn't draw over those parts again and instead headed off to complete the rest of the image.  Given a selection of random near by points to scribble over to next it would need to decide which one was best based on how dark the point needed to be, how close the point was, and if it had been there before, and how many times.

Anyway this was the first result... a bit busy, not very accurate, and not much economy of line.  Very clearly not drawn by a human, or at least not by a sober one.



Next I decided to pre-process the images in Photoshop with an edge-detect filter.  I was going to write my own fast difference of gaussians (or even simpler a difference between neighbours) but I wanted to check the principle worked first.



Straight away much more economical with the lines, and to some extent more accurate.  Since edges or lines essentially have direction and fills don't (ie. you can shade a fill by moving the pen in any direction you want until you reach a border but you can only run a pen along a line in one of two directions) I could use some code to make the line prefer to not change direction and keep heading straight ahead.  This saves some of the over and back-ness of the free scribbling algorithm but isn't perfect.  My next step is to get the scribbler to intelligently work out which way the edge (or line) that it is currently drawing is curving, and to anticipate where to look next.  I think that when we draw we tend to look ahead a fair bit and queue up drawing instructions so our eyes are always a few centimetres ahead of our pen.  If our eyes get lost and have to search then our pen pauses for a moment.

Next I compared the two computer drawing algorithms with my own human 'blind' rotoscope.  The video is here if you want to see it.

Despite my algorithm's improvements it still seems like it misses important features in the image and doesn't prioritise what to draw.  I think that rather than looking for the darkest areas or the sharpest edges it probably needs to find a few high contrast points in the image and link them via as many different edges as are available.  I then built a homemade gaze tracker with my bike helmet, a webcam, AfterEffects, an OpenOffice Spreadsheet, Python and Blender.

The calibration here isn't perfect but this next video demonstrates an important fact; that the eye makes big jumps across the image and doesn't smoothly follow any lines, unless they are moving.  It also doesn't scan around anywhere near as much as I was expecting.  It avoids the edges of the image - I assume because these can be seen in periphery anyway and the film-maker probably hasn't put anything of interest there.  I'm also quite amazed by how much the moving street scene causes the gaze to get pushed out the right edge of the screen - the scribbler also suffered from that quite a bit on the moving images.  When the gaze goes out of the film window that's usually a blink, not a calibration problem.

Well that's by some stretch the longest post yet and almost makes up for a quiet month on the blog.  Now for some maths...

Optical Flow and Rotoscopy

Been feeling pretty ill the last few days. Managed to drag myself to the Broadcast Video Expo yesterday at Earls Court. Would love to go again tomorrow to watch some of Arri's lighting seminars, and the DaVinci Resolve colour grading workshops, but probably won't be feeling good enough yet so it'll be another day at home storyboarding and tidying up odds and ends.



Last week I rotoscoped over the intro to 'Down by Law' with one long continuous virtual line using blender. At college Felipe suggested I could do a bit more with the data and maybe find some way of using the video to distort the drawings. I wrote a python script using the OpenCV bindings for Python 2.7 to analyse the optical flow of the original film footage. I based the script on the lkdemo using CalcOpticalFlowPyrLK. The script dumped out a load of text files containing the feature tracking info.

In blender I wrote a second script which warped my 'drawing' (made up of vertices and edges) on a frame by frame basis using the nearest tracked feature point to drag the line around. Effectively the script puts the line's control points (vertices) into voronoi cells which are shifted by the tracked features (like a virtual earthquake). I added in very simple outlier detection checking consistency with the flow of neighbouring points along the rotoscoped line. I had wanted to convert the point cloud of feature track points to a mesh using delaunay triangulation, and then use the animated mesh to deform the string of vertices, but it would probably have been excessively slow in python!

Anyways thanks to Felipe for pushing me that bit further. Meanwhile progress on the scribbler continues - I've added in three more drawing styles (two of them shown below) and a depth of field control. Next step is to work on edge detection (I've got down on paperware how it should work) and adding a bit of stability to make sure no essential parts of an image get left undrawn on any one frame.