Maya User Framebuffer Tips

User Framebuffers can be useful for splitting up a render into several component passes at render time, to be used later for compositing. Version 3.3 of mental ray (Maya 6) allows up to 8 user framebuffers, but the functionality is not enabled from the Maya UI. Complementary shaders that know how to write into these framebuffers are required to take full advantage of this workflow, but it may not be as hard as you think.

First, lets talk about how to do this with mi files and vanilla mental ray standalone. It requires:

• putting the user framebuffer statements in the options block of the mi file.
• putting matching output statements in the camera block.
• using the mi_fb_put/get in your material shaders to access the framebuffers.

In the current versions of Maya, by default there is no UI to setup the framebuffers in the option block, and output statements in the camera block. However, there is a mechanism called Custom Text that allows you to specify such things when you export mi files. Be careful, this doesn't mean you can interactively render from Maya and get the same behavior. That takes a little more work, which we'll talk about further below. First, thanks to Andy Lomas, we'll show some example MEL scripting that implements the Custom Text method.

Custom Text method

As Andy states, "Here's some mel that I've used in the past with Maya 5.0 to set up framebuffer output using mrmaya. It might need a bit of modification for Maya 6.0, but the principle should work.":

// setup framebuffers in options block using miText

string $textNodeA = `createNode mentalrayText` ;
setAttr ($textNodeA+".mode") 1 ;
setAttr ($textNodeA+".text") -type "string"
( "      frame buffer 0 \"+rgba_fp\"\n" +
"      frame buffer 1 \"+rgba_fp\"\n" +
"      frame buffer 2 \"+rgba_fp\"\n" +
"      frame buffer 3 \"+rgba_fp\"\n" +
"      frame buffer 4 \"+rgba_fp\"\n" +
"      frame buffer 5 \"+rgba_fp\"\n" +
"      frame buffer 6 \"+rgba_fp\"\n" );
connectAttr -f ($textNodeA+".message") mentalrayGlobals.miTextOptions;

// setup framebuffers on camera using miText

string $textNodeB = `createNode mentalrayText`;
setAttr ($textNodeB+".mode") 1;
setAttr ($textNodeB+".text") -type "string"
( "      output \"fb0\" \"iff" "buf0.iff\"\n\" +
"      output \"fb1\" \"iff\" \"buf1.iff\"\n\" +
"      output \"fb2\" \"iff\" \"buf2.iff"\n\" +
"      output \"fb3\" \"iff\" \"buf3.iff"\n\" +
"      output \"fb4\" \"iff\" \"buf4.iff"\n\" +
"      output \"fb5\" \"iff\" \"buf5.iff"\n\" +
"      output \"fb6\" \"iff\" \"buf6.iff"\n\" );
addAttr -ln miText -at message perspShape;
connectAttr -f ($textNodeB+".message") perspShape.miText;

Now you may notice that those are floating point framebuffers, and he's only showing use of 7 of 8 available. But keep in mind that these are samples, which get filtered into pixels between the framebuffer and output image stage. So your image output data can be specified as either integer or floating point RGB.

Now you can use mi_fb_put in your material shaders. The framebuffer statement automatically initializes them to 0 for each sample. Usually, you would test to see if this is the primary material shader, the one the eye ray first hits, and just write to the framebuffer(s) if that is the case. Obviously, this could get quite complicated depending on the desired complexity of your rendering layers.

Interactive method

In order to make this happen interactively from Maya, you can enable the framebuffers by hacking a hidden feature, and then write a shader that does the equivalent of setting up the output statements. Some folks have hacked this into a geometry shader. Or, since it can have access to all the framebuffers, an output shader might be able to write some pass files for interactive use. More on this later...

The hidden feature which enables the framebuffers in Maya 6 is in a mental ray AE template file called AEmentalrayOptionsTemplate.mel. This should be located in mayadir/scripts/AETemplates/. In that file, uncomment the line near the bottom that says miOptionsFramebufTemplate. Now there will be a Framebuffer section when you select miDefaultOptions. The UI for selecting the framebuffer type is a bit funky, which is why it was probably commented out, but you could double check that you selected the correct type, by exporting mi files.