{"id":21971,"date":"2012-06-15T14:04:36","date_gmt":"2012-06-15T14:04:36","guid":{"rendered":"http:\/\/www.labri.fr\/perso\/barla\/blog\/?p=21971"},"modified":"2014-02-12T17:35:53","modified_gmt":"2014-02-12T17:35:53","slug":"mls-gradient-fields-aftermaths","status":"publish","type":"post","link":"https:\/\/www.labri.fr\/perso\/barla\/blog\/?p=21971","title":{"rendered":"MLS-GF aftermaths"},"content":{"rendered":"<p id=\"top\" \/><strong>Edge-aware smoothing:<\/strong> Use post-fitting orientation mismatch as a way to reject pixels as in Bilateral Filter, and warp the kernel according to the local gradient field curvature. Try that in PixelBender!<\/p>\n<p><strong>Volume visualization<\/strong>: Apply in 3D using additional data to clamp support and feed to a transfer function. Present that to Chuck Hansen or JP Domenger so that they incorporate it in their software.<\/p>\n<p><strong>Painterly rendering<\/strong>: use flows computed from 2.5D videos to guide paint strokes or extend gestures in the image. Use confidence to manage paint coverage. Give that to David so that he can play with it! Would be nice in TV-Paint&#8230;<\/p>\n<div id=\"_mcePaste\" style=\"position: absolute; left: -10000px; top: 0px; width: 1px; height: 1px; overflow: hidden;\">\n<h3>Applications<\/h3>\n<ul>\n<li><strong>2D\/3D integration<\/strong>: Cheap texture and reflection mapping, ideal for integrating video footage inside 3D worlds, via compositing solutions. Could be pottentially interesting inside Nuke or Pachwork3D.<\/li>\n<li><strong>Inverse lighting<\/strong>: Very good first guess for drawing highlights in isolated locations, which can then be reproduced with apropriate source lights. Could be interesting in Modo, as part of the SLIT kit.<\/li>\n<li><strong>2D animation<\/strong>: apply to interpolated (vector) or frame-by-frame (raster) animated normal maps. Use tracking to cary anchor points. Potential use in Nuke and TV Paint.<\/li>\n<li><strong>Logo creation: <\/strong>Start from scratch using either nDo2, or diffusion curves, compare to a classic pipeline in Photoshop. Apply it on manao\/maverick logos !<\/li>\n<li><strong>Image editing: <\/strong>roughly paint depth in photoshop (ex: for a building), compute normals by differentiation, create masks to apply different materials, and experiment with the changing of texture, reflections, materials, etc. Best done in Photoshop!<\/li>\n<\/ul>\n<h3>Projects:<\/h3>\n<h4>Vector shading<\/h4>\n<p>A direct extension of surface flows would consist in deforming vector primitives for the easy drawing of shading or texture. With the current system, this is not easily done: deformations modify topology and do not preserve pure 2d characteristics (like smoothness). With a regularisation technique, one could draw silhouettes (possibly adding cross curves like in cross shade), that would define normal and depth maps, that would deform drawn primitives while at the same time keeping their 2D smoothness and connectivity.<br \/>\nIt would permit to deform brush paths and add stylization in post-process, hence achieving interesting shading abstractions. Or it could be used with diffusion curves, to create smooth shading (and fake materials).<br \/>\nThis is similar in spirit to the perspective drawing tool in Illustratorr, which could also be used to add one more level of indirection: one could then move a vector drawing in perspective, adjust its shape and material effects, all in a dynamic manner.<\/p>\n<h4>Expressive compositing<\/h4>\n<p>We talked about this one for a while: here, the idea is mainly to modify surface shape in the rendering of a 3D scene.<br \/>\nWe assume we have different rendered layers, and we will have to modify them so that shape appears to have changed.<br \/>\nCompared to the work we have submitted, here we have to deal with existing materials and lighting.<br \/>\nThus the goal is not to blur or deform an arbitrary image, but to re-use existing information (what and how to re-use is the key issue).<br \/>\nThis is in spirit closer to &#8220;Image-based material editing&#8221;, but differs in two ways: we have access to layers, but we must work with videos!<\/p>\n<h4>Shading flows<\/h4>\n<p>When working on the differential analysis, I first tried to write it in terms of simple materials to simplify Xavier&#8217;s equations.<br \/>\nSo I&#8217;ve decided to use a material that&#8217;s a mix between Lambertian and Specular (using a Fresnel term for blending).<br \/>\nI nearly got it until the end (I need to find derivatives of Spherical Harmonics for that), but already got plenty of insights!<br \/>\nI really have to present you these results, because it shows that diffuse and specular effects depict shape in quite different ways !<br \/>\nThis is very interesting regarding perception, and I wonder whether we could submit that to a Vision (or Optics?) journal.<br \/>\nBut this is also very promising for CG: it tells you precisely how to modify lighting to best reveal shape, both through diffuse and shiny reflections!<br \/>\nThis would be an interesting come back to the Light Warping and Radiance Scaling techniques, but could also be the route to Sculptural Shading (at last!).<\/p>\n<h4>Material appearance<\/h4>\n<p>The two first projects are mainly concerned with the manipulation of shape (given material&amp;lighting) or lighting (given shape&amp;material).<br \/>\nWhat about the manipulation of material appearance then? Logically, it should be investigated by treating shape&amp;lighting together&#8230;<br \/>\nThinking about it, I believe we already have some clues about that: remember when I&#8217;ve tried to work out a formal version of light-free shading?<br \/>\nWell, actually I had found out that a material could be turned into a convolution kernel in image space, whose parameters depend on shape variations and light source distance&#8230;<br \/>\nThis time Xavier and I have worked out the maths, but the text is not very easy to understand in its current form because there&#8217;s no image to illustrate it&#8230;<br \/>\nBut potentially, it could be a way to redefine material appearance directly in terms of image processing! That would be the come back of BADFs \ud83d\ude42<\/p>\n<div id=\"_mcePaste\" style=\"position: absolute; left: -10000px; top: 8px; width: 1px; height: 1px; overflow: hidden;\">date limite de la prise de ma d\u00e9cision finale<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>Edge-aware smoothing: Use post-fitting orientation mismatch as a way to reject pixels as in Bilateral Filter, and warp the kernel according to the local gradient field curvature. Try that in PixelBender! Volume visualization: Apply in 3D using additional data to clamp support and feed to a transfer function. Present that to Chuck Hansen or JP &#8230; <a title=\"MLS-GF aftermaths\" class=\"read-more\" href=\"https:\/\/www.labri.fr\/perso\/barla\/blog\/?p=21971\" aria-label=\"Read more about MLS-GF aftermaths\">Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[641],"tags":[],"class_list":["post-21971","post","type-post","status-publish","format-standard","hentry","category-tech"],"_links":{"self":[{"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=\/wp\/v2\/posts\/21971","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=21971"}],"version-history":[{"count":10,"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=\/wp\/v2\/posts\/21971\/revisions"}],"predecessor-version":[{"id":34081,"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=\/wp\/v2\/posts\/21971\/revisions\/34081"}],"wp:attachment":[{"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=21971"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=21971"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.labri.fr\/perso\/barla\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=21971"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}