technical architecture.

Recursive Tiff to PDF

by Dave Buchhofer on Apr.01, 2010, under Admin, Batching, utility

Another one of those semi random requests that turned out to be more of a pain in the arse than it really should be. the mission: Converting a large mass of scanned tiff CAD drawings into multi page PDF’s

There are two ways to do this fairly quickly, the easy way if you don’t have many individual sets to work with is to simply drag all the tif images into the window in Acrobat Pro and save the file from there.

If however, like us, you have to convert 42 gigs or so of tiff’s spread across 204 folders and group the PDF’s by the folders, you can programmatically do it via some batch scripting and the use of a few free command line utilities from the internet.

First thing to do is convert each individual TIF file to a PDF, for this, we use Imagemagick this is a tool that can do many things but we’re really just going to abuse the ‘convert’ application for now.

After installation, you can then in the dos prompt type ‘convert file.tif file.pdf’

For our purposes we want to convert ALL of the files, so we can set it up in a recursive for loop using

Saved into a file “Convert all Tiffs in Subdirectory to PDFs.bat” or something equally exciting:
FOR /R %%a IN (*.tif) DO convert "%%a" "%%a.pdf"

After that is done we will use a second utility to ‘join’ the pdf’s the PDF toolkit

With this util we can modify pdf files quickly; we’ll use the Append function

pdftk *.pdf cat output combined.pdf

This will concatenate all pdf’s in the current directory and write them into ‘combined.pdf’ in the same directory. Great!

To expand this into a large subdirectory we do a little more batch scripting

The below is saved into a file “Join all PDF in directory tree.bat”

FOR /F "delims=" %%a in ('DIR /S /B /AD ^|SORT') DO ( CD "%%a" pdftk *.pdf cat output "%%a_joined.pdf" )

This will output a combined PDF for each of the individual directories.

A good time was had by all.

Imagemagick looks to have quite a few useful applications and will warrant some more exploration soon.

1 Comment more...

Interactive Lab Demo

by Dave Buchhofer on Feb.10, 2010, under Architecture, Interactive, Unity3D, csharp

Here is another Architectural interactive test project. This time, a laboratory setup, a tighter interior space, set up without any real lighting (1 direct light + ambient colors + Ambient Occlusion currently) created with the goal of customizing the unity asset pipeline to be a little more friendly for architectural work.

First, the Project, roughly 6mb, clicking on the screenshot will open the project in a new window. its another quickly evolving work in progress, Warning: No consideration has been taken currently for speed on older hardware, ~100fps on an 8800GT was the target.

Some findings and random ramblings on arch and interactive 3d:

unlike with general gaming, with architecture you are in most cases going to already have a model provided in some form or another. odds are, the model is going to be pretty horrific, as arch models tend to be built for the purpose of either:
- A: Viz… the Viz model will require a lot of cleanup in terms of material work, and stripping down a lot of the high poly fluff required for rendering.
- B: Design Development… the DD model is normally plagued more by bad modeling conventions, no naming, poor materials, materials not fully assigned, or mangled geometry due to being stretched and squished for days on end to keep up with a design.
you will probably still want to break up the file by some logical layer type system, by default this will land you with a unique material for each material of each fbx file
a small AssetPostProcessor script that works on the OnMaterialAssign function can make the process of sharing materials across multiple FBX files much easier to handle, with the added bonus of being able to easily progressively build up a library of reusable realtime materials

 
//file: MaterialsPostProcessor.cs
//goal: Share a library of materials across different assets based on the name of material supplied in the FBX file
// Dave Buchhofer - 02.10.2010
 
using UnityEngine;
using UnityEditor;
using System.Collections;
 
public class MaterialsPostProcessor : AssetPostprocessor
{
    Material OnAssignMaterialModel (Material material, Renderer renderer)
    {
        //The path where you keep your "Standard" library of materials
        string StandardMatPath = "Assets/DMGStandardMaterials/" + material.name + ".mat";
 
        //the path where you want to keep your "temp" materials, that weren't found above
        //So you have a logical place to look for materials that need editing and tweaking for realtime work.
        string TemporaryMatPath = "Assets/Mesh/Materials/" + material.name + ".mat";
 
        //Check for it in the standard 
        if (AssetDatabase.LoadAssetAtPath(StandardMatPath, typeof(Material)))
        {
            Debug.Log("FOUND: " + StandardMatPath);
            return (Material)AssetDatabase.LoadAssetAtPath(StandardMatPath, typeof(Material));
        }
        //Else check to see if we've already built one in the temp path
        if (AssetDatabase.LoadAssetAtPath(TemporaryMatPath, typeof(Material)))
        {
            Debug.Log("FOUND temp: " + TemporaryMatPath);
            return (Material)AssetDatabase.LoadAssetAtPath(TemporaryMatPath, typeof(Material));
        }
        //Or create it?
        material.shader = Shader.Find("Diffuse");
        AssetDatabase.CreateAsset(material, TemporaryMatPath);
        Debug.Log("CREATED temp: " + TemporaryMatPath);
        return material;
    }
}

also check out the previous arch experiment a few posts down: Basketball Arena

1 Comment :Architecture, editorscripting, Unity3D, workflow more...

Recursive file size listing output to excel

by Dave Buchhofer on Dec.30, 2009, under Admin, Batching, Scripting, vbscript

Today I had to find all JPG files over a certain size in a large directory tree of a few thousand images, to find some poorly compressed jpg’s, fix the compression, and replace them on a website. Todays cavaet, the CMS is pretty weak and only lets you update 1 image at a time, so you can’t just blindly resave everything and reupload, todays Second cavaet, I dont have server level access to said website, so all the normal automated ways are out of luck.

joy.

So I found a little snip of code online, and threw a pretty simple hack job together to narrow the scope some. (PS: If you do end up doing any vbscript, I’d suggest VbsEdit for an IDE)

Anyway! the code:

It searches through a directory tree of your choosing
for any file matching the filetype: “jpg”
that is above a size threshold: fileSizeThreshold (In this case, 2kb.. so essentially everything)
and outputs it into an excell sheet with full path and size information
so that you can easily Sort, Filter, Delegate, or use as data for further automating!

' // **************************************
' //    ComputerHighGuys recursive search
' //     
' //    Date Created: 20 Aug 07
' //	http://www.tek-tips.com/faqs.cfm?fid=6716
' //
' //	Adjusted to a jpg file search and 
' //	added excel output for easy sorting/filtering
' //	
' // **************************************
 
' // If we'd like to save the output into an excel sheet
WriteExcel = "True"
 
Dim objexcel
excelRow=2
 
' // value the filesize needs to exceed to be visible in the output
fileSizeThreshold=2
 
If WriteExcel = "True" Then
	' // Create the Excel sheet to drop the information into
	Set objExcel = createobject("Excel.application")   
	objexcel.Workbooks.add
	objexcel.Cells(1, 1).Value = "Folder Name"
	objexcel.Cells(1, 2).Value = "Filename"
	objexcel.Cells(1, 3).Value = "Filesize"
	objexcel.Cells(1, 4).Value = "Filesize Unit"
	objexcel.Visible = True
	Wscript.Sleep 300
End If
 
' // Directory to search
searchDir = "C:\"
 
set objFSO=CreateObject("Scripting.FileSystemObject")
Set objFolder = objFSO.GetFolder(searchDir)
Set colFiles = objFolder.Files
 
' // Launch the function
ScanSubFolders(objFolder)
 
' // recursive function that will search through all subfolders for a specified
' // filetype and output some information about the files to an Excel Sheel
Sub scanSubFolders(objFolder)
	' // Grab sub folders    
    Set colFolders = objFolder.SubFolders
 
    For Each objSubFolder In colFolders
 
    	' // the files to search
        Set colFiles = objSubFolder.Files
        For Each objFile in colFiles
 
            ' // the extension of the filetype to search for  
            If lcase(Right(objFile.Name,3)) = "jpg" Then
 
                ' // Getting File size in KB
                If round(objFile.Size/1024,1) > fileSizeThreshold Then
 
                	' // echo the files to the console
                	WScript.Echo objSubFolder.Path & "  " & objFile.Name & "  " & round(objFile.Size/1024,1)  & "KB"
 
					' // write various shit to excel~                    
                    If WriteExcel = "True" then
                        wscript.Echo objSubFolder.Path & "  " & objFile.Name & "  " & round(objFile.Size/1024,1)  & "KB"
                    	objexcel.Cells(excelRow, 1).Value = objSubFolder.Path
    					objexcel.Cells(excelRow, 2).Value = objFile.Name
    					objexcel.Cells(excelRow, 3).Value =  round(objFile.Size/1024,1)
    					objexcel.Cells(excelRow, 4).Value =  "KB" 
    					excelRow=excelRow+1    
                    End If
                End if    
            End If
        Next
        ScanSubFolders(objSubFolder)
    Next
End Sub

2 Comments :Admin, asset management, Gallery, Scripting, utility, VB more...

Architectural Interactive Demo

by Dave Buchhofer on Dec.12, 2009, under Architecture, Interactive, Unity3D

Here’s a current work in progress Interactive Architectural type demo of a college Basketball Arena

it hasn’t been terribly optimized yet, so it requires a fairly hefty video card to play smoothly, that said, it runs at ~150fps on my 8800GT here, but at 5fps on my parents 3 year old dell. So your mileage may vary while I experiment!

http://www.buchhofer.com/upload/files/labs/drexel/ ~5mb

Everything has been done with ingame lighting and shaders, nothing is baked, and was meant as a test specifically for that purpose, to see how far it can be pushed before requiring Texture Baking on a fair sized scene. (With the aim to be able to quickly iterate models in earlier phases of design without having to unwrap and bake!)

To the game! There is a quality button on the lower left that toggles between various settings of Anti Aliasing, Shader Quality, Shadow Quality.. so you can tailor it to a point to your hardware.

On the upper left is a selection of preset Camera views, clicking and dragging the mouse button in the viewport anywhere will ‘look’ the camera from the preset view.

On the upper right is some fun stuff, toggles for Ambient occlusion, Bloom shaders (glow) and a couple of sliders to adjust those primary values

there is also an alternate Daylighting scheme, if you do this i suggest toggling the roof, then you can adjust the time of day with the ‘TimeOfDay’ slider, and the ambient lighting with the ambient.

the rest are just toggles to show/hide various layers of objects

3 Comments more...

python based model/texture/matlib browser

by Dave Buchhofer on Jul.24, 2009, under Uncategorized

This was a learning experiment a few months back that showed some promise, and turned into a pretty useful script, I debated passing it around due to its current ‘raw’ness, but it has already been very helpful here, and I think some feedback may motivate me to tighten it up some more!

DOWNLOAD: pyAssetBrowser.zip

Setting it up

You pick a ‘Root Directory’ by editing the settings.ini file and changing the “rootPath=X:\\Model_Library\\” to match up with your chosen path. ex: if your model library is on C:\Assets, then you will make the settings.ini file read as: “rootPath=C:\\Assets\\” every directory under this ‘root’ directory will show up in the Treeview interface on the left hand side.

(and here comes the part that made me wary of distributing it)

For the connection with 3dsmax to work, python needs to know which version of max to connect to when ‘Max.Application’ is invoked! the cleanest way to do this is to just run Adams script: registerPythonCom.ms, this should work for most people! the cases I’ve found where it doesn’t work, are where there are more restrictive windows user permissions, in which case it may be required to manually do this. I’ve included a couple example .reg files that can connect to max 2008 / 2009, all that’s needed to suit them to your particular installation is to edit the paths under the LocalServer32 reg key for your particular max version.

On the 3dsmax side of things, there are a couple scripts included, the pyAssetbrowser.ms script is a Struct including some simple functions for things like importing, merging, converting to vrmesh, billboarding image maps, collapsing, re-linking maps using a few stripped down bits of modified code from an older version of ColinSenner’s Relink Bitmaps script

File Structure

for the browser to work right as is, there are a few ‘format’ type issues that are needed to make it even remotely useful.

To use it as a Model Library:
- A directory that includes .Max files.
- If there is a .JPG file with the same name as the .MAX file, then it will be displayed as the thumbnail for the item, if there is no preview jpg the maxfile will show with a 3dsmax icon.

The material library setup is kind of strange, but its modeled somewhat after easily adding items from the VrayMaterials.de site, by just unzipping the library and ensuring that the directory has the same name as the thumbnail.

Material Library:
- A Directory of .JPG thumbnails (Any size)
- In the same directory, a Folder with the same name as the .JPG thumbnail
- in the named folder (Yea, this gets a bit messy) a 3dsmax .mat material library file (Any name), and any textures required

Billboard/texture Library:
- A directory of images, Most formats should work, Tiff’s wont show thumbnails due to some open source licensing silliness with PIL i believe.
- One of the good things here, is that it can build a thumbnail list very very fast. not quite picasa fast, but far quicker than explorer does.

I’m sure there’ll be many issues, so call it a beta, and let me know if anyone has feedback!

I’m including the source also, as 90% of it is all bits and pieces of other peoples work!

A little source info

I compiled with python 2.5
the awesome ThumbnailCtrl from Andrea Gavanna
- I did some *very* slight modifications on it to allow it to show .max files as an icon
used PIL 1.1.6+
Requires pyWin32 v212+ for the COM bridging to talk to 3dsMax
Interface in wxwidgets, using wxPython 2.8
‘lazy’ directory treeview from here
exe compiled with py2exe 0.6.9

Some things that currently aren’t possible, but are obvious additions:

Multiple Root items in the treeview to allow for the fact of the various libraries to be in seperate locations/drives…

This can be done, but is a bit of a pain in the arse, how you would do it would be to create a virtual root, which is hidden (As the current root item is now) and add in your sub items below it with specific paths, the trick is just to adjust the path sent to the thumbnail control to account for the ‘virtual’ root. Not difficult, but possibly a little time consuming, so its been shelved unless someone wants to buy me a beer or two to polish it up

There is currently no way to retrieve the explorer bitmap thumbnails for .max files through python/pywin32, there is a few hints at there being possible workarounds and ideas that could work on Mark Hammond’s mailing list, but most of the talk about this is several years stale! any thoughts on this from the python heads would be more than welcome!

there are a few errors that pop up after closing but they dont affect usage currently.

3 Comments more...

ServerUtilities v.85

by Dave Buchhofer on Jun.23, 2009, under Batching, maxscript, vray

Its been a while since I’ve posted anything, so its time for the habitual blog sorry/script update!

We recently picked up VrayRT, and i found myself again vnc’ing over to farm machines to restart services and well, that just wont do!

So I did a small update to the ServerUtilities script adding a few services, and doing a little work on the underbelly of it to make it a bit more stable. also packaged it into an installer because I was getting a few people that had just installed it strangely and were having issues.

screenshot and download link

Download the Installer

3 Comments more...

QuickCollapse.ms

by Dave Buchhofer on Mar.16, 2009, under Batching, maxscript

Here’s partial results from last nights maxscript silliness.

QuickCollapse.ms

QuickCollapse is a maxscript Struct with a few functions to speed up collapsing large numbers of objects, it provides feedback in the listener window so you can see progress as it goes.

Run script will trigger ‘CollapseSelected’ on your current selection. there isn’t much Error Checking involved, so filter your selection for meshes, and remeber that it SHUTS UNDO OFF for the collapse so that we dont have RAM overruns on large objectsets, so save/hold first eh?

14 Comments more...

Revit->FBX->Max, Collapsing large numbers of meshes

by Dave Buchhofer on Mar.15, 2009, under Architecture, Batching, Scripting, maxscript, utility

Well, lets just say that dealing with large Revit files can get a little ugly for visualization purposes to say the least. there are several fundamental issues currently, ranging from workflows in building usable familys inside of revit, to dealing with the geometry after the fact for rendering.

I had the pleasure to again deal with a fairly large Revit model. It only weighed in at about 300mb to start! for working on final renderings thats all well and good, and not a real issue to deal with, but for mid project progress renders it can get a bit painful doing a lot of the deconstruction needed to make it useful to work in in 3dsMax. spending several hours ‘cleaning’ a revit model in max so that you can get any sort of rendering done is lets say, a bit depressing, when you know you’ll have to do the same process again in 2 weeks.

all that said, I started looking for some solutions to one small facet of the problem, and found some good case study testing on efficiently attaching a ton of meshes done by Dave Stewart and also a fair number of tips from the Maxscript crew at CGTalk

So I did a small adaptation of Dave’s attachment script above, which can be found here: CollapseSelected-inParts5.ms Its not pretty, but we’ll get there soon enough.

It takes your current selection, and simplifies the number of objects by the square root (Dave’s tested optimal amount for speed collapsing!) its a work in progress, and i figure i’ll take this, combined with set of other tools for collapsing either by Selected Material, Similar Objects+Instances, Layer, and some name filtering. that should take the day long revit cleanup jobs and compress them down to an hour or so.

Yay.

2 Comments more...

Mass Exporting, again

by Dave Buchhofer on Feb.24, 2009, under Batching, maxscript, utility

Mass exporting..

We have a fun little virtools app that loads in various NMO’s exported out from max on the fly as needed based on user input, so in the process of working out what our process would be, we came to the need to have each seperate 3dsmax hierarchy exported out into a seperate virtools NMO file.

Pretty simple, but the question came up on a forum today and its been a useful little script that I’ve been using for years now.

the guts are:

 
--wanky recursive function to parse hierarchy into an array
fn addChildrenToArray theChildren currentObjsToExport =
(
	for c in theChildren do
	(
		append currentObjsToExport c
		addChildrenToArray c.children currentObjsToExport
	)
)
 
fn massExportfn hier path filetype =
(	
 
	exportSelection = selection as array
	if hier == true then
	(
		baseNodesToExport = for o in exportSelection where o.parent == undefined collect o
	)
	else baseNodesToExport = exportSelection
 
	--parse through said root nodes
	for o in baseNodesToExport do 
	( 
		oldPos = o.position
		o.position = [0,0,oldPos.z]
 
		--children returns a 'NodeChildrenArray, so convert that to an array manually.
		--include the current node in the array no matter what.
		currentObjsToExport = #(o)
 
		-- if we're packaging hierarchys then collect childrem
		if hier == true do
		(
			addChildrenToArray o.children currentObjsToExport
		)
 
		select currentObjsToExport		
 
		--random info for use to ogle at the export.. yea yea
		format "\tExport:\t%\tas\t%\n"  o.name filetype
		format "\tTo:\t%\n" (path + "/" + o.name + filetype)
		format "\n-----------------------------\n\n"
 
		-- if we need to save selected use this export type
		if filetype == ".MAX" then
		(
			savenodes currentObjsToExport (path + "/" + o.name + filetype) quiet:true
		)
		else
		(
			--export using the name of the root node as the filename
			exportfile (path + "/" + o.name + filetype) #noPrompt selectedOnly:true 
		)
		o.position = oldPos
	)
	select exportSelection
)
 
-- and its used by:
massExportfn true "C:/temp/" ".vmo"

Theres an interface for it here, but i’ve got it built into many other small little pipeline specific utils, so that may not be of any real use to anyone else, but this little snip might be useful for someone out there.

When Sebbys Attack.

by Dave Buchhofer on Feb.09, 2009, under Uncategorized

A shot from earlier today, I’m pretty sure there’s a few more solid ones in there, once I plow through the immense stack of fresh RAW photos clogging up my system.

On the plus side, after a few days of shooting i definitely have a much more solid grasp of ISO vs Aperture vs. Shutter, so I’m learning well why the first few days of shooting were horrifically grainy and blurry.

f5 to f11, ISO as low as it’ll go and still let me handhold the shutter speed!

Leave a Comment :Babys, Photo, Photography more...

« Older Entries