Over the past few years we have been building and refining a data platform for AEC and Infrastructure projects – we call it Deep Space. It has allowed us to manage huge projects, develop powerful analytics, and increase productivity and efficiency with automations.

We just released a brand new version of our Revit addin, check out the first demonstration video here:

Feel free to check out the website at https://www.deepspacesync.com/

We hope to meet you in Deep Space soon!

I have developed a working BIM360 to on-premises backup mechanism. There were a few different ways to go…

 

I initially considered:

  • Hacking Windows Explorer to touch the files in the BIM 360 node and try and trigger the BIM360 Docs download and copy to local PC or network location
  • Developing a Forge app that essentially pulls the desired models out of the BIM 360 cloud to desired location?

In the end, I decided to use the Revit API (Dynamo and Python), along with VirtualBuiltApp, to essentially reverse engineer a folder structure from the local CollaborationCache folder. These ideas are hinted at here.

 

In simple terms, the BIM 360 Docs on-premises backup workflow is:

  1. Create a super federated BIM 360 model (with all other BIM 360 models linked into it)
  2. A Dynamo script reloads all links in order to cache them locally
  3. The same script interrogates, then determines the target file name and folder from VirtualBuiltApp
  4. Dynamo then copies the files into their ‘backup’ location on the local network

 

Some more detail is presented below:

 

A) Cache Cleaner CMD Script (may not really be necessary, as the ‘reload’ should overwrite superseded cache anyway):

Kill Revit if Open… be careful of below steps, particularly if Revit has crashed recently. You might need these local copies 🙂

cd "%LOCALAPPDATA%\Autodesk\Revit\Autodesk Revit 2018"
cd CollaborationCache
rmdir oldCache
mkdir oldCache
for /d %i in (*) do move "%~i" oldCache

 

B) Manual steps to setup Revit model environment to run the script:

  1. Open Revit
  2. Open Federated BIM 360 file (worksets closed)
  3. Unload all links
  4. Open all Worksets

 

C) Dynamo Script:
Run Dynamo (Python) script that reloads and unloads all links (this collects .rvt into cache). Key Python commands to use are: RevitLinkType.Load() and RevitLinkType.Unload(None).

Potential Problems:

  • Coordination Monitor alert, no longer exists
  • Instance of link needs Coordination Review
  • “Some numerical data within the imported file was out of range. This numerical data has been truncated.”
  • “Geometry in the file … has extents greater than 20 miles (33km).”

Local cached Models appear into

%LOCALAPPDATA%\Autodesk\Revit\Autodesk Revit 2018\mainCacheFolder\hostModelIdFolder\LinkedModels\*.rvt

 

D) Dynamo Script continues to:
Rename and copy to desired target location. How do we determine this?

Using proprietary VirtualBuiltApp schema:

  1. Use known directory data along with known JSON data for the BIM 360 files (harvested previously by our proprietary systems)
  2. Build list of target paths
  3. Run copy from Dynamo to copy to target network directory

Part of the script:

 

Note: you can actually browse some BIM 360 Docs data in a .sqlite3 file located in path shown below. You can use DB Browser for SQLite as a viewer.

Previously, this was all working with the old Desktop Connector and b360provider.exe (I think?) but the new versions of Desktop Connector seem to have broken the ability to ‘browse’ C4R Revit models on BIM 360… and guess what? There is no Solution. This functionality is in fact ‘by design’. You can copy and paste a non-C4R Revit model to BIM 360 Docs via the connector, and that RVT will be visible in Windows Explorer. But if you have C4R initialised Revit models on BIM 360 Docs, they will not be visible under the BIM 360 (Preview) node in Windows Explorer. You can only view those C4R Revit files in their folder structure by navigating through Revit 2018.3 or newer.

This is the feedback from Autodesk:

Revit cloud models (C4R models) are not supported in Desktop connector. They won’t be showing up in Desktop Connector. Published C4R models are only available in Revit and Docs.


For C4R models, published models will have all versions in BIM 360. This will be recommended workflow to manage models:
https://knowledge.autodesk.com/support/revit-products/learn-explore/caas/CloudHelp/cloudhelp/2016/ENU/Revit-CAR/files/GUID-AE6DBCDF-3DB1-41CB-9DC5-B0C336E6C851-htm.html

You can also save copies of the model locally to put in your backup system. Also from each user’s machine, C4R cache will be another source of backups. 

Also refer to How to backup Revit cloud models

I went looking for another answer, but there really isn’t one. Below is a list of steps I took to play around with the Desktop Connector installation. In the end, the only way I can foresee being able to ‘automate’ onsite BIM 360 Docs backups will be to leverage the CollaborationCache folder. You will need some Revit API know-how to forcibly populate this through some BIM 360 Reload methods (refer here).  Unfortunately this will be a flat list of files without descriptive names, so there will have to be some workarounds and cleanup scripts in place (we already have a system of BIM 360 file management in VirtualBuiltApp).  I intend to post some more about these workarounds in future.

Keep reading for excessive detail on my Desktop Connector journey …

___________________________________________________________________________________

I used Glary Utilities Uninstall tool:

Entry shown:

"C:\ProgramData\Package Cache\{b46602fe-a358-48c4-a0e0-905010df08e2}\DesktopConnector.exe" /modify

Uninstalled this old version. Interestingly, there were 3 different uninstall processors triggered? Like I had three different versions installed?

 

 

Also tried removing this folder and reinstalling:

I found a pile of errors in the diagnostic logs:

CDX pipleline builder output
Output=Warnings received from AddInStore.Rebuild
========================
Could not connect up a part in a pipeline to its neighbors: AddInAdapter Name: "ISecureSettingsProviderViewToContractAddInAdapter" Location: "AddInSideAdapters\Comet.AddInFramework.DataSourceProviders.ProviderAdapters.dll".

and a few of these

========================
Could not connect 2 valid add-in model parts.
========================

========================
While inspecting an assembly, caught a BadImageFormatException: The file is not a valid binary: C:\Program Files\Autodesk\Desktop Connector\AddIns\ADriveConnector\AdWebServices.dll This occurred while inspecting assembly C:\Program Files\Autodesk\Desktop Connector\AddIns\ADriveConnector\AdWebServices.dll.
========================

========================
While inspecting an assembly, caught a BadImageFormatException: The file is not a valid binary: C:\Program Files\Autodesk\Desktop Connector\AddIns\ADriveConnector\AdWebServicesUI.dll This occurred while inspecting assembly C:\Program Files\Autodesk\Desktop Connector\AddIns\ADriveConnector\AdWebServicesUI.dll.
========================

This note seemed useful:

Starting CDX pipline builder: C:\Program Files\Autodesk\Desktop Connector\Comet.Applications.PipelineBuilder.exe C:\Program Files\Autodesk\Desktop Connector

 

I scanned through the Registry and deleted old references to Desktop Connector.

 

Found this link with the command line switches: here

Tried an install from elevated command prompt:

C:\Temp>DesktopConnector-x64.exe /install /quiet

Note: you do not actually need the Desktop Connector to collaborate on BIM 360 Revit projects in the cloud. You only need Revit 2018.3 or newer, and the relevant license for Revit Cloud Collaboration (too hard to keep track of the names but I think ‘BIM 360 Design’ is the current terminology).

You will be able to see this link in Revit 2018.3 or newer:

When ‘linking’ Revit models, CAD files, and other resources, you will see this option when working in a cloud workshared model:

These models will be temporarily stored in the CollaborationCache folder:

I previously posted about how to quickly repath links based on some control mechanisms. Enter BIM 360, and the wild world of Revit cloud worksharing… I expect that it will be commonplace now for existing projects and datasets to move across to BIM360 ‘mid project’. But that creates some interesting problems, like creating folders, dealing with the initiation process, and replacing local Revit Links with their cloud versions.

This post is focused on that process of changing all of the Revit link paths to link to the BIM 360 models. Unfortunately, the previous method I used (TransmissionData, like eTransmit) is not available for cloud hosted models. So how do we automate this process?

We went about it this way:

  1. Initiate all Revit models on the BIM 360 Document Management cloud (manually, for now)
  2. Create one federated model on the BIM 360 cloud that links in all the other cloud hosted Revit models. You might do this one manually, using Reload From in the Manage Links dialog box.
  3. Once you have that one ‘super host model’, use a batch process to harvest all of the cloud model data
  4. Using the harvested data, create a script that implements a Reload From method to batch reload local models from their cloud counterpart

On the journey to solving step 3, I experimented with a few different methods. I discovered that you need to use the ExternalResource class to get information about BIM 360 cloud models (not ExternalReference).

I also realised that I had to deal with Reference Information, which appears to be a .NET dictionary per link that stores some funky Forge IDs and so on. But I want to store all this data in our VirtualBuiltApp BIM Management system, so I had to serialise the Reference Information to a string that could be stored in a database VARCHAR field (or push to Excel if you are still doing things the old way). Dimitar Venkov gave me a few tips about using JSON with IronPython in Dynamo (thanks mate!), so after that all the harvesting pieces were in place!

Here is some of the harvesting and JSON code. Notice that I played around with using a container class to pass data between Dynamo nodes. In the end, JSON string was the answer:

 data = []
for u in unwraps:
    data.append(u.GetExternalResourceReference(linkresource))

class dummy(object):
    def ToString(self):
        return 'container'
container = dummy()

sdicts = []

for y in data:
    dictinfo = ExternalResourceReference.GetReferenceInformation(y)
    container.dictinfo = dictinfo
    infos.append(container)
    shortnames.append(ExternalResourceReference.GetResourceShortDisplayName(y))
    versionstatus.append(ExternalResourceReference.GetResourceVersionStatus(y))
    insessionpaths.append(y.InSessionPath)
    serverids.append(y.ServerId)
    versions.append(y.Version)
    sdicts.append(json.dumps(dict(dictinfo)))

The next step was to create the ‘batch reload from’ tool. Now that we had the necessary data, we just had to use it to grab the matching cloud path information (from our database) and apply it to each Revit link.

I created a node that essentially built a new reference path from the JSON and other data that we had harvested. Here is some of that code:

 des = []
for x in referencesInfo:
    des.append(json.loads((x)))
newdicts = []
for y in des:
    newdicts.append(Dictionary[str, str](y))

serverGuids = []
for g in serverIdsIn:
    tempguid = Guid(g)
    serverGuids.append(tempguid)

newrefs = []
for z in range(len(referencesInfo)):
    serverIdIn = serverGuids[z]
    referenceInfo = newdicts[z]
    versionInfo = versionsInfo[z]
    sessionPathIn = sessionsPathIn[z]
    tempRef = ExternalResourceReference(serverIdIn, referenceInfo, versionInfo, sessionPathIn)
    newrefs.append(tempRef)

OUT = newrefs

The final step was to get a RevitLinkType and a matching ReferenceInformation and apply them to each other. I stored the data in our cloud based BIM Management Application, VirtualBuiltApp. Then I could easily just pull the data into Dynamo with a suitable database connector, and match up the RevitLinkType in the current file with its associated cloud identity. For that genuine 90s feel, you could use Excel to store the data as it is just a JSON string and some other strings:

Here is the key bit of code that actually changes the link path (without all of my other error checking bits and pieces):

     try:
        newCloudPath = newCloudPaths[l]
        reloaded = fileToChange.LoadFrom(newCloudPath, defaultconfig)
        successlist.append(reloaded.LoadResult)
        TransactionManager.Instance.ForceCloseTransaction()
    except:
        successlist.append("Failure, not top level link or workset closed")

To actually implement the script and get productive, I opened 4 instances of Revit, and then used this process in each instance:

  1. Open the Revit file from BIM 360, with Specify… all worksets closed
  2. Unload all links
  3. Open all worksets
  4. Run the Reloader Script
  5. Confirm link status in Manage Links
  6. Optional: Add ‘bim 360 links loaded’ text to Start View (just for tracking purposes)
  7. Optional: Add comment to VirtualBuiltApp (optional, for tracking purposes)
  8. Close and Sync

In this way I can have 4 or more sessions operating concurrently, fixing all the link paths automatically, and I just need to gently monitor the process.

One nice thing is that I set the script up to immediately Unload a link after it had obtained and applied the new Path information. This means that the Revit instance does not get bogged down with many gigs of link data in memory, and in fact this is way faster than trying to use Manage Links for a similar process.

Ideally I would like to fully automate this, to the point where it opens each file, runs the script, and syncs. Unfortunately, time didn’t allow me to get all the code together for that (for now).

Finally, because we are using our custom built schema and validation tools, we can easily create visuals like this:

Modified versions of the Dynamo graphs can be found on the Bakery Github here:

dyn folder