I am pretty sure that sawdust float in water so you could dump the mixture in water so that all of the sawdust float on the surface while the nails sink to the bottom.
You might also be able to use a strong magnet since nails are usually magnetic while sawdust is not.
I hope this helps. Let me know if anything is unclear or if you need a different method.
