LDraw into Rhino 3D

LEGO models can be described in the LDraw format. The file format was developed by James Jessiman in 1995. Since then this has become a defacto standard for creating LEGO models on computers.

This program rhildraw is a simple importer of LDraw files. A model can be described by one or more files. We'll get to the details of these files as we go on our journey to implementing the importer.

Overview of the program

The following diagram shows the main structure of the script

flowchart TD
    subgraph init
    A[[Initialization]]
    B[Read file]
    end
    subgraph geom [Read Module]
    direction LR
    C[[Handle commands]]
    D[[Add part geometry]]
    E[[Create block definition]]
    end
    A --> B
    B --> geom
    C --> D
    D --> C
    D --> E

Initialization

We will be using a couple of dictionaries to keep track of files, materials and so on. These will make finding data easier as the command to import another file uses the file name as the key either without or with a parent folder prepended.

Dictionaries for the follow data are used:

<<initialize global variables>>= 


vfiles: Mapping[str, LDrawFile]= dict()
idefs : Mapping[str, InstanceDefinition]= dict()
materials : Mapping[str, LDrawMaterial]= dict()
vertidx = 0

Reading color definitions

Colors are defined just like parts and assemblies. Line type 0 is used through the Colour Definition Language Extension (specification).

For this script we'll be using the colors defined in LDConfig.ldr from the LDraw complete library, although there are alternatives available. Technically the file with standard color definitions contains the following line in its header:

0 !LDRAW_ORG Configuration UPDATE YYYY-MM-DD

But we'll forgo this check and just assume the file we specify is the correct file.

Preparing the colors means preparing color data for materials so that is what we'll be doing.

We need just one simple method to set up the materials dictionary that will contain all the defined colors, and that is the <<method to load colors>>.

<<initialize color definitions>>= 
<<method to load colors>>

To load the colors we just need to get the LDrawFile instance for LDConfig.ldr and pull out the commands. Then we loop over all the commands and handle those that start with the required 0 !COLOUR string. Again we assume that a command is well-formed. For each color command we <<read color properties from command>>, create an instance of LDrawMaterial and add it to the materials dictionary.

<<method to load colors>>= 
def load_colors():
    <<load color commands>>
    COLOR_CMD = "0 !COLOUR "
    TO_REMOVE = "0 !"
    for cmd in cmds:
        if cmd.startswith(COLOR_CMD):
            <<read color properties from command>>
            <<create material and save to dictionary>>
    print("Colors read")

To <<load color commands>> we simple get the LDrawFile instance for LDConfig.ldr and use get_commands() on that.

<<load color commands>>= 
colorldr = get_ldraw_file("LDConfig.ldr")
cmds = colorldr.get_commands()

Then for each line we clean up the part we're not using (0 !). Most keys have values, but for finishes there is a little snag. Most of the finishes, CHROME, PEARLESCENT, RUBBER, MATTE_METALLIC and METAL when specified make this an odd list. To even that out we will add a dummy value to the list, so we can simple iterate over the list in pairs and generate a dictionary out of it. Eventually we'll be able to properly handle finishes. Perhaps MATERIAL SPECKLE and MATERIAL GLITTER with the use of some good procedural texturing.

<<read color properties from command>>= 
properties = dict()
cmd = cmd[len(TO_REMOVE):]
cmd_split = cmd.split()
if len(cmd_split) % 2 == 1:
    cmd_split.append('dummy')
keyvalue_count = len(cmd_split) // 2
for i in range(0, keyvalue_count*2, 2):
    properties[cmd_split[i]] = cmd_split[i+1]

With the properties now converted to a dictionary we can instantiate an LDrawMaterial and add it to the main materials dictionary with the CODE value as key. This key we can later use during handling of other commands that specify a key code to find the correct entry.

<<create material and save to dictionary>>= 
ldraw_material = LDrawMaterial(properties)
materials[properties["CODE"]] = ldraw_material

LDrawMaterial class

<<LDraw material class>>= 
class LDrawMaterial:
    def __init__(self, props):
        self.properties = props
        self.name = props["COLOUR"]
        self.render_material = None

    def _get_color4f(self, colstr):
        colstr = colstr[1:]
        r = int(colstr[0:2], 16) / 255.0
        g = int(colstr[2:4], 16) / 255.0
        b = int(colstr[4:6], 16) / 255.0
        return Color4f(r, g, b, 1.0)

    def _alpha(self, alphastr):
        alpha = 1.0 - (float(alphastr) / 255.0)
        return alpha

    def get_render_material(self):
        if self.render_material == None:

            raise Exception(f"Material non-existant: {self.name}")
        return self.render_material

    def create_render_material(self):
        for rm in sc.doc.RenderMaterials:
            if rm.Name == self.name:
                self.render_material = rm
                return
        pbr_rm = RenderContentType.NewContentFromTypeId(pbr_guid)

        _basecolor = self._get_color4f(self.properties["VALUE"])

        _roughness = 0.2
        _metallic = 0.0
        _opacity = 1.0

        if "ALPHA" in self.properties:
            _opacity = self._alpha(self.properties["ALPHA"])
            _roughness = 0.03

        if "METAL" in self.properties or "CHROME" in self.properties:
            _metallic = 1.0
            _roughness = 0.03
        if "MATTE_METALLIC" in self.properties:
            _metallic = 1.0
            _roughness = 0.3

        pbr_rm.SetParameter(PbrNames.BaseColor, _basecolor)
        pbr_rm.SetParameter(PbrNames.Opacity, _opacity)
        pbr_rm.SetParameter(PbrNames.Metallic, _metallic)
        pbr_rm.SetParameter(PbrNames.Roughness, _roughness)

        pbr_rm.Name = self.name
        self.render_material = pbr_rm
        sc.doc.RenderMaterials.Add(pbr_rm)
<<initialize global variables>>=+ 
pbr_guid = ContentUuids.PhysicallyBasedMaterialType

Reading files

All LDraw files consist of lines where each line is of a certain line type. Each line type denotes a command. All LDraw files have these line types, so it is easy to parse them.

LDraw files have the extenions .mpd, .ldr and .dat. These extensions are typically used for files with specific usage although from just the contents it doesn't really matter.

However, the following are well-established conventions. The .dat file is used to define geometry of parts and subparts. The .ldr files often contain commands to load parts, they express assemblies or models. The .mpd is a form of multi-part document, where in one file several files are combined.

In our script we represent LDraw files with the class LDrawFile. The class itself is very simple. It has only a method for <<initialization of LDrawFile>> and a method to <<get commands from LDrawFile>>.

<<LDrawFile class>>= 
class LDrawFile:
    <<initialization of LDrawFile>>
    <<get commands from LDrawFile>>

    def __repr__(self):
        if self.is_3dm():
            return f"LDrawFile '{self.name}', a Rhino 3d file."
        else:
            return f"LDrawFile '{self.name}', {len(self.get_commands())} commands."

    def is_ccw_winding(self):
        if self.is_3dm():
            return
        cmds = self.get_commands()
        for cmd in cmds:
            if cmd.startswith('0') and 'BFC' in cmd and 'CCW' in cmd:
                return True

        return False

    def is_3dm(self):
        return self.suffix == '.3dm'

Instantiation of an LDrawFile instance takes a required parameter path. From this Path instance the file name, suffix and filename with parent are extracted.

If the contents of the file are known during instancing the data can be passed in as a list of strings.

<<initialization of LDrawFile>>= 
def __init__(self, path : Path, data : List[str] = []):
    self.commands = data
    self.path = path
    self.name = path.name
    self.suffix = path.suffix
    self.pname = f"{path.parent.name}\\{self.name}"
    self.ppname = f"{path.parent.parent.name}\\{path.parent.name}\\{self.name}"

For most files the contents will not be known in advance. When retrieving the contents the file gets lazily loaded and the contents cached in case this particular file is used again in a model.

Files are to be read in as UTF-8 files. At some point it would probably be good to perhaps support falling back to encodings of old files.

Reading of the file also strips already out empty lines.

<<get commands from LDrawFile>>= 
def get_commands(self):
    if len(self.commands)==0 and not self.is_3dm():
        with self.path.open(encoding="utf-8") as f:
            cmds = [l.strip() for l in f.readlines()]
            cmds = [c for c in cmds if len(c) > 0]
            self.commands = cmds

    return self.commands

Line types

The heart of the LDraw specification revolves around line types. The specification essentially is a collection of lines whereby the first character of the line tells us what kind of operation has to happen.

For now we'll keep the importer simple and go where the path is easiest. We'll implement mainly line types 3, 4 and 1, with a smidge of 0 through the color definition language. Eventually we may want to include also 2 and 5 lines, but for just importing complete models for set dressing and rendering purposes this should get us going.

Line types 3 and 4 add polygon methods, triangles and quads respectively.

Adding polygons

Handling line types 3 and 4 means adding either a triangle or a quad. The form of line types 3 and 4 are similar. A line type three starts with the number 3 followed by a color code and three triplets of values that are either integers or floating point. For both line types we ignore the color code as we'll be using the color codes defined on line type 1 commands. But we'll get into that later.

The line type four has a similar format, starting with number 4, the color code and then four triplets of values.

From the start number we parse how many triplets - vertices we'll be handling. This is the first character of the command string.

<<parse vertex count>>= 
vertices = int(cmd[0])
elements = vertices * stride
to = start + vertices * stride

Each triplet is a coordinate in the polygon. The order in which the triplets are presented is also the vertex order.

The format also has a facility to tell the winding for polygons, but we ignore that here. We'll be using simple CW winding.

Each line has all their elements separated by white space. A simple split will suffice. We'll take only all parts after the color code. Parse all elements as floats, since they form vertices. If any element fails to parse as a float we'll skip adding this as a polygon.

<<parse vertices>>= 
d = cmd.split()[start:to]
try:
    d = [float(f) for f in d]
except Exception:
    return

With all numbers parsed we can add them as vertices to the mesh, increasing the vertidx count for each item.

<<add vertices to mesh>>= 
vidxs = list()
for i in range(0, elements, stride):
    V = apply_transforms(d[i:i+stride], xforms)
    m.Vertices.Add(*V)
    vidxs.append(vertidx)
    vertidx += 1

if len(vidxs) == 4 and from_winding == winding:
    
    rev = vidxs[1:]
    rev.reverse()
    vidxs = [vidxs[0]] + rev

With all vertices added and vertidx adjusted we can now add the face definition for either the triangle or the quad.

<<add face to mesh>>= 

if vertices == 4:
    mf = MeshFace(vidxs[0], vidxs[1], vidxs[2], vidxs[3])
elif vertices == 3:
    mf = MeshFace(vidxs[0], vidxs[1], vidxs[2])
if from_winding != winding and False:
    
    mf = mf.Flip()
    
m.Faces.AddFace(mf)

To bring it all together we can say that to add a polygon from a command string we first <<parse vertex count>> and <<parse vertices>>, then <<add vertices to mesh>> followed by the final step <<add face to mesh>>.

<<add polygon method>>= 
def add_poly(m : Mesh, cmd : str, xforms : list, from_winding = Winding.CCW, winding = Winding.CCW):
    global vertidx
    stride = 3
    start = 2
    <<parse vertex count>>
    <<parse vertices>>
    
    <<add vertices to mesh>>
    <<add face to mesh>>

Loading a model

flowchart
    A[Model]
    B[Assembly]
    C[Part]

    A --> B
    B --> B
    B --> C
    C --> C

TO BE WRITTEN AS LITERATE PROGRAM

<<the whole script.*>>= ./src/rhildraw.py $
import traceback
import sys
import scriptcontext as sc
import math

from typing import Mapping, List

import Rhino

from Rhino.Geometry import Transform, Mesh, Vector3f, Point3f, MeshFace
from Rhino.Display import Color4f
from Rhino.DocObjects import ObjectAttributes, ObjectMaterialSource
from Rhino.DocObjects import InstanceDefinition
from Rhino.Render import ChildSlotNames, ContentUuids, RenderContentType

PbrNames = ChildSlotNames.PhysicallyBased
rhmath = Rhino.RhinoMath

from System.IO import DirectoryInfo, Directory, File, FileInfo
from System.IO import EnumerationOptions, SearchOption

from pathlib import Path
from enum import Enum

class Winding(Enum):
    CW = 1
    CCW = 2

    def is_ccw(self):
        return self == Winding.CCW
        

    def is_cw(self):
        return self == Winding.CW
        

    def flip(self):
        return Winding.CW if self == Winding.CCW else Winding.CCW

<<LDrawFile class>>
<<LDraw material class>>

<<initialize global variables>>

def refresh():
    sc.doc.Views.Redraw()
    Rhino.RhinoApp.Wait()

class LegoXform:
    def __init__(self, data : str):
        data = data.strip()
        if len(data) > 0:
            try:
                d = [float(f) for f in data.split()[2:14]]
                self.x = d[0]
                self.y = d[1]
                self.z = d[2]
                self.a = d[3]
                self.b = d[4]
                self.c = d[5]
                self.d = d[6]
                self.e = d[7]
                self.f = d[8]
                self.g = d[9]
                self.h = d[10]
                self.i = d[11]
                xform : Transform = Transform.Identity
                xform.M00 = self.a
                xform.M01 = self.b
                xform.M02 = self.c
                xform.M03 = self.x
                xform.M10 = self.d
                xform.M11 = self.e
                xform.M12 = self.f
                xform.M13 = self.y
                xform.M20 = self.g
                xform.M21 = self.h
                xform.M22 = self.i
                xform.M23 = self.z
                self.xform = xform
            except Exception as e:
                self.xform = Transform.Identity
        else:
            self.xform = Transform.Identity

    def set_xform(self, xform):
        self.xform = xform

    def transform_point(self, u : float, v : float, w : float):
        p = Point3f(u, v, w)
        p.Transform(self.xform)
        return [p.X, p.Y, p.Z]

    def get_xform(self):
        return self.xform


rhino_orient = LegoXform("")
rhino_orient.set_xform(
    Transform.Rotation(
        rhmath.ToRadians(-90.0),
        Vector3f.XAxis,
        Point3f.Origin
    )
)
id_xform = LegoXform("")

def clean_name(part_name):
    part_name = part_name.removesuffix(".dat")
    part_name = part_name.removesuffix(".DAT")
    part_name = part_name.removesuffix(".ldr")
    part_name = part_name.removesuffix(".LDR")
    return part_name

def prepare_parts_dictionary():
    global lib_path, vfiles
    library_path : Path = Path(lib_path)
    library_path_net : DirectoryInfo = DirectoryInfo(lib_path)
    all_parts_net = library_path_net.EnumerateFiles("*", SearchOption.AllDirectories)
    for p in all_parts_net:
        fn = Path(p.FullName)
        if fn.suffix.lower() in ('.txt', '.zip', '.exe', '.DS_Store'):
            continue

        if len(fn.suffix) > 0 and fn.suffix.lower() in ('.3dm'):
            print(f'Adding {fn}')

        ldrawfile = LDrawFile(fn)
        vfiles[ldrawfile.name] = ldrawfile
        vfiles[ldrawfile.pname] = ldrawfile

        if ldrawfile.is_3dm():
            print(f'\t{ldrawfile} .. [{ldrawfile.name}] :: {vfiles[ldrawfile.name]}')

def prepare_idefs_dictionary():
    for idef in sc.doc.InstanceDefinitions:
        idefs[idef.Name] = idef

def update_idefs_dictionary(part_name):
    idef_part_name = clean_name(part_name)
    idef = sc.doc.InstanceDefinitions.Find(idef_part_name)
    if idef:
        idefs[idef_part_name] = idef
        return idef
    return None

def apply_transforms(v, xforms):
    for xform in xforms:
        v = xform.transform_point(*v)
    return v

def collate_transforms(xforms):
   xform = Transform.Identity
   for _xform in xforms:
       xform = xform * _xform.get_xform()

   return xform

<<add polygon method>>

first_time = True
def get_ldraw_file(part_name : str) -> LDrawFile:
    global vfiles, first_time

    part_name = part_name.replace('/', '\\')

    part_3dm = Path(part_name).with_suffix('.3dm').name

    print(f"...searching for [{part_3dm}]")

    if "springMesh" in part_3dm:
        pass

    if part_3dm in vfiles.keys():
        print(f"\t found {part_3dm}")
        return vfiles[part_3dm]

    if part_name in vfiles.keys():
        return vfiles[part_name]

    raise Exception(f"Part file not found: {part_name}")


def load_part(part : LDrawFile, m : Mesh, xforms : list, override_invert_faces = False):
    cmds = part.get_commands()
    invert_faces = False
    direction = Winding.CCW if part.is_ccw_winding() else Winding.CW
    if part.is_ccw_winding() and not override_invert_faces:
        invert_faces = True
    elif part.is_ccw_winding() and override_invert_faces:
        invert_faces = False
    elif override_invert_faces and not part.is_ccw_winding():
        invert_faces = True

    

    invert_next = False
    for cmd in cmds:
        if is_invert_cmd(cmd):
            
            invert_next = True
            continue
        if cmd.startswith('1'):
            d = cmd.split()
            xform = LegoXform(cmd)
            prt = ' '.join(d[14:])
            _xforms = [xform] + xforms[:]
            try:
                part_file = get_ldraw_file(prt)
            except Exception:
                print(f"\tERR: Failed getting part {prt}, skipping")
                continue
            load_part(part_file, m, _xforms, invert_next)
        elif cmd.startswith('3') or cmd.startswith('4'):
            direction_to_use = direction
            if override_invert_faces:
                direction_to_use = direction.flip()
            add_poly(m, cmd, xforms, direction, direction.flip() if override_invert_faces else direction) 
        invert_next = False

def get_part_idef(prt):
    p = Path(prt)
    pname = clean_name(p.name)
    if pname in idefs:
        return idefs[pname]

    return None

def is_invert_cmd(cmd):
    return cmd.startswith('0') and 'BFC' in cmd and 'INVERTNEXT' in cmd

def contains_poly_commands(cmds):
    for cmd in cmds:
        if len(cmd) == 0: continue
        if cmd[0] in ("2", "3", "4", "5"):
            return True

    return False

def blockinstance_for_idef(prt, _xforms, obattr, override_invert_faces=False):
    idef = get_part_idef(prt)
    xform = collate_transforms(_xforms)
    if idef == None:
        add_part(prt, override_invert_faces)
        idef = update_idefs_dictionary(prt)
    if idef != None:
        sc.doc.Objects.AddInstanceObject(idef.Index, xform, obattr)
    else:
        print(f"Failed to add part {prt}")

def load_model_part(part : LDrawFile, xforms : list, override_invert_faces = False):
    cmds = []
    cmds = part.get_commands()

    direction = Winding.CW
    flipped = False

    cmds = [l for l in cmds if len(l)>0]
    if part.is_ccw_winding():
        
        direction = Winding.CCW

    for cmd in cmds:

        if is_invert_cmd(cmd):
            
            direction = not direction
            override_invert_faces = True
            flipped = True
            continue
        if cmd.startswith('1'):
            d = cmd.split()
            color_code = d[1]
            materials[color_code].create_render_material()
            rm = materials[color_code].render_material
            xform = LegoXform(cmd)
            prt = ' '.join(d[14:])
            _xforms = xforms[:] + [xform]

            obattr = ObjectAttributes()
            obattr.Name = clean_name(prt)
            obattr.Visible = True
            obattr.MaterialSource = ObjectMaterialSource.MaterialFromObject
            obattr.RenderMaterial = rm

            if prt.lower().endswith(".ldr"):
                ldr_file = get_ldraw_file(prt)
                if contains_poly_commands(ldr_file.get_commands()):
                    add_part(prt, override_invert_faces)
                    idef = update_idefs_dictionary(prt)
                    if idef != None:
                        xform = collate_transforms(_xforms)
                        sc.doc.Objects.AddInstanceObject(idef.Index, xform, obattr)
                    else:
                        print(f"Couldn't add part {prt}")
                elif ldr_file.is_3dm():
                    blockinstance_for_idef(prt, _xforms, obattr, override_invert_faces)
                else:
                    load_model_part(ldr_file, _xforms, override_invert_faces)
            else:
                blockinstance_for_idef(prt, _xforms, obattr, override_invert_faces)
                """
                idef = get_part_idef(prt)
                xform = collate_transforms(_xforms)
                if idef == None:
                    add_part(prt, override_invert_faces)
                    idef = update_idefs_dictionary(prt)
                if idef != None:
                    sc.doc.Objects.AddInstanceObject(idef.Index, xform, obattr)
                else:
                    print(f"Failed to add part {prt}")
                """

            
            if flipped:
                
                direction = not direction
                flipped = False
            refresh()

def add_virtual_file(model : Path, filename : str, data : List[str]):
    global vfiles
    virtual_file_path = model.parent / 'virtual' / filename
    virtual_file = LDrawFile(virtual_file_path, data)
    vfiles[virtual_file.name] = virtual_file
    vfiles[virtual_file.pname] = virtual_file


def load_model(model : LDrawFile):
    lines = model.get_commands()

    FILE_START = '0 FILE '
    first_file = ''
    if model.suffix.lower() == '.mpd':
        
        cur_file = ''
        file_data = []
        for l in lines:
            if l.startswith(FILE_START):
                if cur_file != '':
                    add_virtual_file(model.path, cur_file, file_data)
                if cur_file == '':
                    first_file = l[len(FILE_START):]
                cur_file = l[len(FILE_START):]
                file_data = [l]
            else:
                file_data.append(l)
        add_virtual_file(model.path, cur_file, file_data) 
    elif model.suffix.lower() == '.ldr':
        first_file = model.name
    start_part = get_ldraw_file(first_file)

    load_model_part(start_part, [rhino_orient])


def add_part(part_name : str, invert_faces = False):
    global vertidx
    vertidx = 0
    name = clean_name(part_name)

    print(f"adding part {part_name} ({name})")

    existing_idef = sc.doc.InstanceDefinitions.Find(name)
    if existing_idef:
        print(f"\tSkipping {part_name}, instance already created")
        return
    else:
        pass 

    ldraw_file = get_ldraw_file(part_name)

    if ldraw_file.is_3dm():
        print(f"... reading 3dm file {ldraw_file.path}")
        f3dm = Rhino.FileIO.File3dm.Read(f'{ldraw_file.path}')
        obs_to_add = list()
        attrs_to_add = list()
        for ob in f3dm.Objects:
            if ob.Geometry.ObjectType == Rhino.DocObjects.ObjectType.InstanceReference:
                continue
            obs_to_add.append(ob.Geometry)
            attrs_to_add.append(ob.Attributes)

        obattr = ObjectAttributes()
        obattr.Name = name
        obattr.Visible = True
        obattr.MaterialSource = ObjectMaterialSource.MaterialFromParent
        print(sc.doc.InstanceDefinitions.Add(
            f'{name}',
            f'High-definition Rhino version of {name}.',
            Rhino.Geometry.Point3d.Origin,
            obs_to_add,
            attrs_to_add
        ))
        f3dm.Dispose()
    else:
        mesh = Mesh()
        obattr = ObjectAttributes()

        obattr.Name = name
        obattr.Visible = True
        obattr.MaterialSource = ObjectMaterialSource.MaterialFromParent

        load_part(ldraw_file, mesh, [id_xform], invert_faces)
        mesh.Weld(math.radians(50))
        mesh.Normals.ComputeNormals()
        mesh.Compact()

        if mesh.Vertices.Count > 0 and mesh.Faces.Count > 0:
            sc.doc.InstanceDefinitions.Add(obattr.Name, "", Point3f.Origin, mesh, obattr)

<<initialize color definitions>>










lib_path = "/Users/jesterking/Documents/brickdat/ldraw"


prepare_parts_dictionary()
prepare_idefs_dictionary()
load_colors()










fl : Path = vfiles["42064-1.mpd"]









fl : Path = vfiles["8836-1.mpd"]


load_model(fl)

refresh()

sc.doc.Views.EnableRedraw(True, True, True)

Rhino.RhinoApp.RunScript("ZEA", False)

print("Done")