technical architecture.

Another one of those semi random requests that turned out to be more of a pain in the arse than it really should be. the mission: Converting a large mass of scanned tiff CAD drawings into multi page PDF’s

There are two ways to do this fairly quickly, the easy way if you don’t have many individual sets to work with is to simply drag all the tif images into the window in Acrobat Pro and save the file from there.

If however, like us, you have to convert 42 gigs or so of tiff’s spread across 204 folders and group the PDF’s by the folders, you can programmatically do it via some batch scripting and the use of a few free command line utilities from the internet.

First thing to do is convert each individual TIF file to a PDF, for this, we use Imagemagick this is a tool that can do many things but we’re really just going to abuse the ‘convert’ application for now.

After installation, you can then in the dos prompt type ‘convert file.tif file.pdf’

For our purposes we want to convert ALL of the files, so we can set it up in a recursive for loop using

Saved into a file “Convert all Tiffs in Subdirectory to PDFs.bat” or something equally exciting:
FOR /R %%a IN (*.tif) DO convert "%%a" "%%a.pdf"

After that is done we will use a second utility to ‘join’ the pdf’s the PDF toolkit

With this util we can modify pdf files quickly; we’ll use the Append function

pdftk *.pdf cat output combined.pdf

This will concatenate all pdf’s in the current directory and write them into ‘combined.pdf’ in the same directory. Great!

To expand this into a large subdirectory we do a little more batch scripting

The below is saved into a file “Join all PDF in directory tree.bat”

FOR /F "delims=" %%a in ('DIR /S /B /AD ^|SORT') DO (
CD "%%a"
pdftk *.pdf cat output "%%a_joined.pdf"
)

This will output a combined PDF for each of the individual directories.

A good time was had by all.

Imagemagick looks to have quite a few useful applications and will warrant some more exploration soon.

Here is another Architectural interactive test project. This time, a laboratory setup, a tighter interior space, set up without any real lighting (1 direct light + ambient colors + Ambient Occlusion currently) created with the goal of customizing the unity asset pipeline to be a little more friendly for architectural work.

First, the Project, roughly 6mb, clicking on the screenshot will open the project in a new window. its another quickly evolving work in progress, Warning: No consideration has been taken currently for speed on older hardware, ~100fps on an 8800GT was the target.

Some findings and random ramblings on arch and interactive 3d:

• unlike with general gaming, with architecture you are in most cases going to already have a model provided in some form or another. odds are, the model is going to be pretty horrific, as arch models tend to be built for the purpose of either:
  • A: Viz… the Viz model will require a lot of cleanup in terms of material work, and stripping down a lot of the high poly fluff required for rendering.
  • B: Design Development… the DD model is normally plagued more by bad modeling conventions, no naming, poor materials, materials not fully assigned, or mangled geometry due to being stretched and squished for days on end to keep up with a design.
• you will probably still want to break up the file by some logical layer type system, by default this will land you with a unique material for each material of each fbx file
• a small AssetPostProcessor script that works on the OnMaterialAssign function can make the process of sharing materials across multiple FBX files much easier to handle, with the added bonus of being able to easily progressively build up a library of reusable realtime materials

 
//file: MaterialsPostProcessor.cs
//goal: Share a library of materials across different assets based on the name of material supplied in the FBX file
// Dave Buchhofer - 02.10.2010
 
using UnityEngine;
using UnityEditor;
using System.Collections;
 
public class MaterialsPostProcessor : AssetPostprocessor
{
    Material OnAssignMaterialModel (Material material, Renderer renderer)
    {
        //The path where you keep your "Standard" library of materials
        string StandardMatPath = "Assets/DMGStandardMaterials/" + material.name + ".mat";
 
        //the path where you want to keep your "temp" materials, that weren't found above
        //So you have a logical place to look for materials that need editing and tweaking for realtime work.
        string TemporaryMatPath = "Assets/Mesh/Materials/" + material.name + ".mat";
 
        //Check for it in the standard 
        if (AssetDatabase.LoadAssetAtPath(StandardMatPath, typeof(Material)))
        {
            Debug.Log("FOUND: " + StandardMatPath);
            return (Material)AssetDatabase.LoadAssetAtPath(StandardMatPath, typeof(Material));
        }
        //Else check to see if we've already built one in the temp path
        if (AssetDatabase.LoadAssetAtPath(TemporaryMatPath, typeof(Material)))
        {
            Debug.Log("FOUND temp: " + TemporaryMatPath);
            return (Material)AssetDatabase.LoadAssetAtPath(TemporaryMatPath, typeof(Material));
        }
        //Or create it?
        material.shader = Shader.Find("Diffuse");
        AssetDatabase.CreateAsset(material, TemporaryMatPath);
        Debug.Log("CREATED temp: " + TemporaryMatPath);
        return material;
    }
}

also check out the previous arch experiment a few posts down: Basketball Arena

Today I had to find all JPG files over a certain size in a large directory tree of a few thousand images, to find some poorly compressed jpg’s, fix the compression, and replace them on a website. Todays cavaet, the CMS is pretty weak and only lets you update 1 image at a time, so you can’t just blindly resave everything and reupload, todays Second cavaet, I dont have server level access to said website, so all the normal automated ways are out of luck.

joy.

So I found a little snip of code online, and threw a pretty simple hack job together to narrow the scope some. (PS: If you do end up doing any vbscript, I’d suggest VbsEdit for an IDE)

Anyway! the code:

It searches through a directory tree of your choosing
for any file matching the filetype: “jpg”
that is above a size threshold: fileSizeThreshold (In this case, 2kb.. so essentially everything)
and outputs it into an excell sheet with full path and size information
so that you can easily Sort, Filter, Delegate, or use as data for further automating!

' // **************************************
' //    ComputerHighGuys recursive search
' //     
' //    Date Created: 20 Aug 07
' //	http://www.tek-tips.com/faqs.cfm?fid=6716
' //
' //	Adjusted to a jpg file search and 
' //	added excel output for easy sorting/filtering
' //	
' // **************************************
 
' // If we'd like to save the output into an excel sheet
WriteExcel = "True"
 
Dim objexcel
excelRow=2
 
' // value the filesize needs to exceed to be visible in the output
fileSizeThreshold=2
 
If WriteExcel = "True" Then
	' // Create the Excel sheet to drop the information into
	Set objExcel = createobject("Excel.application")   
	objexcel.Workbooks.add
	objexcel.Cells(1, 1).Value = "Folder Name"
	objexcel.Cells(1, 2).Value = "Filename"
	objexcel.Cells(1, 3).Value = "Filesize"
	objexcel.Cells(1, 4).Value = "Filesize Unit"
	objexcel.Visible = True
	Wscript.Sleep 300
End If
 
' // Directory to search
searchDir = "C:\"
 
set objFSO=CreateObject("Scripting.FileSystemObject")
Set objFolder = objFSO.GetFolder(searchDir)
Set colFiles = objFolder.Files
 
' // Launch the function
ScanSubFolders(objFolder)
 
' // recursive function that will search through all subfolders for a specified
' // filetype and output some information about the files to an Excel Sheel
Sub scanSubFolders(objFolder)
	' // Grab sub folders    
    Set colFolders = objFolder.SubFolders
 
    For Each objSubFolder In colFolders
 
    	' // the files to search
        Set colFiles = objSubFolder.Files
        For Each objFile in colFiles
 
            ' // the extension of the filetype to search for  
            If lcase(Right(objFile.Name,3)) = "jpg" Then
 
                ' // Getting File size in KB
                If round(objFile.Size/1024,1) > fileSizeThreshold Then
 
                	' // echo the files to the console
                	WScript.Echo objSubFolder.Path & "  " & objFile.Name & "  " & round(objFile.Size/1024,1)  & "KB"
 
					' // write various shit to excel~                    
                    If WriteExcel = "True" then
                        wscript.Echo objSubFolder.Path & "  " & objFile.Name & "  " & round(objFile.Size/1024,1)  & "KB"
                    	objexcel.Cells(excelRow, 1).Value = objSubFolder.Path
    					objexcel.Cells(excelRow, 2).Value = objFile.Name
    					objexcel.Cells(excelRow, 3).Value =  round(objFile.Size/1024,1)
    					objexcel.Cells(excelRow, 4).Value =  "KB" 
    					excelRow=excelRow+1    
                    End If
                End if    
            End If
        Next
        ScanSubFolders(objSubFolder)
    Next
End Sub

Its been a while since I’ve posted anything, so its time for the habitual blog sorry/script update!

We recently picked up VrayRT, and i found myself again vnc’ing over to farm machines to restart services and well, that just wont do!

So I did a small update to the ServerUtilities script adding a few services, and doing a little work on the underbelly of it to make it a bit more stable. also packaged it into an installer because I was getting a few people that had just installed it strangely and were having issues.

screenshot and download link

– .65 – added vray 2010 to services
– .75 – added vrayRT
– .75 – convert to simpler dos prompt
– .75 – added an optional pause to the end of the commandline based tools
– .75 – fixed the invert button in server selection
– .75 – prints the commandline to the listener
– .80 – convert to dos based loop for multiple systems (if pause: hit enter to advance to next machine in loop..)
– .85 – update to support spaces in the max file path! oops!

Download the Installer

Here’s partial results from last nights maxscript silliness.

QuickCollapse.ms

QuickCollapse is a maxscript Struct with a few functions to speed up collapsing large numbers of objects, it provides feedback in the listener window so you can see progress as it goes.

Run script will trigger ‘CollapseSelected’ on your current selection. there isn’t much Error Checking involved, so filter your selection for meshes, and remeber that it SHUTS UNDO OFF for the collapse so that we dont have RAM overruns on large objectsets, so save/hold first eh?

Well, lets just say that dealing with large Revit files can get a little ugly for visualization purposes to say the least. there are several fundamental issues currently, ranging from workflows in building usable familys inside of revit, to dealing with the geometry after the fact for rendering.

I had the pleasure to again deal with a fairly large Revit model. It only weighed in at about 300mb to start! for working on final renderings thats all well and good, and not a real issue to deal with, but for mid project progress renders it can get a bit painful doing a lot of the deconstruction needed to make it useful to work in in 3dsMax. spending several hours ‘cleaning’ a revit model in max so that you can get any sort of rendering done is lets say, a bit depressing, when you know you’ll have to do the same process again in 2 weeks.

all that said, I started looking for some solutions to one small facet of the problem, and found some good case study testing on efficiently attaching a ton of meshes done by Dave Stewart and also a fair number of tips from the Maxscript crew at CGTalk

So I did a  small adaptation of Dave’s attachment script above, which can be found here: CollapseSelected-inParts5.ms Its not pretty, but we’ll get there soon enough.

It takes your current selection, and simplifies the number of objects by the square root (Dave’s tested optimal amount for speed collapsing!) its a work in progress, and i figure i’ll take this, combined with set of other tools for collapsing either by Selected Material, Similar Objects+Instances, Layer, and some name filtering. that should take the day long revit cleanup jobs and compress them down to an hour or so.

Yay.

Mass exporting..

We have a fun little virtools app that loads in various NMO’s exported out from max on the fly as needed based on user input, so in the process of working out what our process would be, we came to the need to have each seperate 3dsmax hierarchy exported out into a seperate virtools NMO file.

Pretty simple, but the question came up on a forum today and its been a useful little script that I’ve been using for years now.

the guts are:

 
--wanky recursive function to parse hierarchy into an array
fn addChildrenToArray theChildren currentObjsToExport =
(
	for c in theChildren do
	(
		append currentObjsToExport c
		addChildrenToArray c.children currentObjsToExport
	)
)
 
fn massExportfn hier path filetype =
(	
 
	exportSelection = selection as array
	if hier == true then
	(
		baseNodesToExport = for o in exportSelection where o.parent == undefined collect o
	)
	else baseNodesToExport = exportSelection
 
	--parse through said root nodes
	for o in baseNodesToExport do 
	( 
		oldPos = o.position
		o.position = [0,0,oldPos.z]
 
		--children returns a 'NodeChildrenArray, so convert that to an array manually.
		--include the current node in the array no matter what.
		currentObjsToExport = #(o)
 
		-- if we're packaging hierarchys then collect childrem
		if hier == true do
		(
			addChildrenToArray o.children currentObjsToExport
		)
 
		select currentObjsToExport		
 
		--random info for use to ogle at the export.. yea yea
		format "\tExport:\t%\tas\t%\n"  o.name filetype
		format "\tTo:\t%\n" (path + "/" + o.name + filetype)
		format "\n-----------------------------\n\n"
 
		-- if we need to save selected use this export type
		if filetype == ".MAX" then
		(
			savenodes currentObjsToExport (path + "/" + o.name + filetype) quiet:true
		)
		else
		(
			--export using the name of the root node as the filename
			exportfile (path + "/" + o.name + filetype) #noPrompt selectedOnly:true 
		)
		o.position = oldPos
	)
	select exportSelection
)
 
-- and its used by:
massExportfn true "C:/temp/" ".vmo"

Theres an interface for it here, but i’ve got it built into many other small little pipeline specific utils, so that may not be of any real use to anyone else, but this little snip might be useful for someone out there.

This handles the services perfectly for me here, I’m doing a little more work on it to support using the server.exe and vrayspawner exe versions as well, but that wont be finished for a while as its a spare time project. I’m really looking for feedback in terms of platforms you’re looking to use it on, and what features would be nice to see in addition.

Click here to Download

Install is pretty straightforward, simply unzip into your Scripts directory.. after a bit more testing for various people I’ll throw together an installer, but for now its slightly manual.

after you unzip it, assuming your max is installed in say, C:\3dsmax08\ you should see several files

Maxscript files: Access them as Maxscript Pulldown, Run Script.
C:\3dsMax08\server_tool_lite_05.mcr — Run this to install the script into the Customize UI, under category dbScripts.
C:\3dsMax08\ServerUtilities\server_tool_lite_05.ms — Run this to try out the tool without installing it into the Customize ui..

C:\3dsMax08\ServerUtilities\… There will be a few other exe’s in this directory, mostly commandline utilities that the script calls to batch modify things.

After you run the script the first time, it will create a file C:\3dsMax08\ServerUtilities\ServerToolLite.ini which will save your settings. In the script, it might be easier for you to add 1 server, and then browse to the ServerToolLite.ini file and manually add the other 19 servers in notepad.

Cheers,
Dave

Looking for a couple testers to beat up on a maxscript for me before I send it out to the whole world.

drop a line to dbuchhofer {AT} gmail {DOT} com if you’ve got a moment to try it out. Things should run much smoother than the old HTA/VBScript nonsense.

Basic Problem: IT has enforced a Screensaver password policy here now Now, with the D3D view port driver, when you come back from a locked workstation, it doesn’t always instantiate the DX display correctly. we’re noticing on our workstations that it will come back working to a point.. IE: All of the geometry on screen is selectable, but only ~20% of it is displayed with edges in max.

a hack to fix it without having to restart max every time the computer locks on you, I’ve found i can Disable and then Enable the Direct3D Cached meshes.. this works.. in a crappy sort of way.

Here’s some code that toggles the d3d cached.. it also helps to show how to access things in max that are not readily open to the maxscript language, by using windows messaging to “push” buttons on dialogs! most credit to Richard at cgtalk!

 
toolTip="toggle the Use Cached D3DXMeshes Viewport"
buttonText="toggle the Use Cached D3DXMeshes Viewport"
 
--start macro
 
-----------------------------------------------------------------------------------------------
--
-- toggles the "Use Cached D3DXMeshes" checkbox of the Viewport configuration
--
-----------------------------------------------------------------------------------------------
 
--diable it while
 dialogMonitorOps.enabled = false
 global prefsDialog_hwnd = undefined
 global retMessage=""
 fn toggleCachedD3DMeshes = (
 
	 --Constants for sendMessage method	
	local BM_GETSTATE = 0xF2
	local BM_CLICK = 0xF5
	local BM_SETCHECK = 0xF1
	local BST_CHECKED = 0x1	 
 
 	local hwnd = dialogMonitorOps.getWindowHandle()
 	local dialogTitle = uiAccessor.getWindowText hwnd
 	if (dialogTitle != undefined) then (
 		if (dialogTitle == "Preference Settings") then (
			prefsDialog_hwnd = hwnd
			format "We're in the preferences dialog\n" 
			local hwnd_children = uiAccessor.getChildWindows hwnd
 			for i = 1 to hwnd_children.count do (
 				local hwnd_child_title = uiAccessor.getWindowText hwnd_children[i]
				if (findString hwnd_child_title "Configure Driver..." == 1) then (
					format "found config button... pressing\n"
					local hwnd_config = hwnd_children[i]
 					uiAccessor.pressButton hwnd_config
					)
				)
			)
		else if (dialogTitle == "Configure Direct3D") then (
			local hwnd_children = uiAccessor.getChildWindows hwnd
 			for i = 1 to hwnd_children.count do (
 				local hwnd_child_title = uiAccessor.getWindowText hwnd_children[i]
				if (findString hwnd_child_title "Use Cached D3DXMeshes" == 1) then (
					format "found the cached button\n"
					local hwnd_cached = hwnd_children[i]
					local CheckState = windows.sendMessage hwnd_cached BM_GETSTATE 0 0
					local IsChecked = bit.get CheckState BST_CHECKED
					format "the checkbox was: %\n" IsChecked
					-- Uncheck it
					if IsChecked then
					(
						windows.sendMessage hwnd_cached BM_CLICK 0 0
						windows.sendMessage hwnd_cached BM_SETCHECK 0 0
						format "Cached D3DXMeshes has been disabled.\n"
						format "Pressing OK on the ConfigureD3D page\n"
						uiAccessor.sendMessageID hwnd #IDOK
						format "Pressing OK on the Preferences page\n"
						uiAccessor.sendMessageID prefsDialog_hwnd #IDOK
						retMessage="Turning the CachedD3DXMeshes *OFF*"
					)
					-- Check it
					else if not IsChecked then
					(
						windows.sendMessage hwnd_cached BM_CLICK 0 0
						windows.sendMessage hwnd_cached BM_SETCHECK 1 0
						format "Cached D3DXMeshes has been enabled.\n"
						format "Pressing OK on the ConfigureD3D page\n"
						uiAccessor.sendMessageID hwnd #IDOK
						format "Pressing OK on the Preferences page\n"
						uiAccessor.sendMessageID prefsDialog_hwnd #IDOK
						retMessage="Turning the CachedD3DXMeshes *ON*"
					)
 				)
 			)
 		)
 	)
	true
 
 )
 
fn toggleUseCache =
(
	dialogMonitorOps.interactive = false
	dialogMonitorOps.unregisterNotification id:#setD3DCache
	dialogMonitorOps.registerNotification toggleCachedD3DMeshes id:#setD3DCache
	dialogMonitorOps.enabled = true
	--run it
	max file preferences
	--disable it!
	dialogMonitorOps.enabled = false
	dialogMonitorOps.unregisterNotification id:#setD3DCache
	--messagebox retMessage
)
 
toggleUseCache()
format retMessage