Wouldn't it be nice if we could automate plugin development for Obsidian? Yeah it would be. A long developer desired-feature for Obsidian has been a command line interface that we haven't been given yet. Now, we do have Obsidian URI's, but they have their limitations... Mainly: - Can only open Vaults that Obsidian has already opened manually - Can't bypass the trust authors dialog - Can't turn off restricted mode - Can't enable plugins So what can we do about it? Well, let's explore the fact that Obsidian is built on Electron and JavaScript. Now Obsidian isn't exactly Open Source, but it isn't exactly Closed Source either. Obsidian's choice to use Electron at its core allows us to explore how Obsidian works. In this Article, I will be exploring how Obsidian works internally, and how you might use this information to automate plugin testing. # Automating Obsidian Internals > [!danger] Disclosure: > This article explores **Obsidian's hidden internals** as a method of automating plugin testing. So, it should go without saying that *this is in no way a statement that the following testing techniques are Obsidian-supported* ## URI handling So we know that currently the only automatic entrypoint into obsidian is to use an Obsidian URI. Now, you would think that Obsidian would allow you to open any vault with the URI `obsidian://open?path=[...]`, but they do not. They only allow you to open vaults that obsidian is tracking. ## The Vault Tracker So, how does obsidian track which vaults have been opened? This was actually easy to find out. The [Obsidian Help Docs](https://help.obsidian.md/Advanced+topics/Using+Obsidian+URI#Action+open) give you a really strong hint as to how they track the vaults internally: ### `open` URI action: > Description: Opens an Obsidian vault, and possibly open a file within that vault. > Possible parameters: > - `vault` can be either the vault name, or the vault ID. > - The vault name is simply the name of the vault folder. > - The vault ID is the random 16-character code assigned to the vault. This ID is unique per folder on your computer. Example: `ef6ca3e3b524d22f`. **There isn't an easy way to find this ID yet, one will be offered at a later date in the vault switcher. Currently it can be found in `%appdata%/obsidian/obsidian.json` for Windows. For macOS, replace `%appdata%` with `~/Library/Application Support/`. For Linux, replace `%appdata%` with `~/.config/`.** That `obsidian.json` is Obsidian's internal vault tracker. If you open the file, you will find that it is a json hashtable. The key of importance is the `vaults` key: ```json { "vaults" : { "[vault_id]" : { "path" : "path/to/vault/root", "ts" : "[13-digit timestamp(milliseconds)]" //"open" : true/false - this key/value pair is only present when it's true } } } ``` Now the Obsidian docs specify that `[vault_id]` is a "randome 16-character code assigned to the vault." However, I have found that this key can be anything (as long as it is a valid JSON key). In fact, I have been setting the `[vault_id]` to be a string containing the git branch name of the Obsidian Plugin and a 14 character timestamp, usually putting that id length well into the twenties. ### Node.js snippet: ```javascript const fs = require( "fs" ) ... const vault_id = "ImA20CharacterString" // sky's the limit here, I've even used special characters and it still works const appdata = `${ process.env.APPDATA || ( process.platform == 'darwin' ? process.env.HOME + '/Library/Application Support' : process.env.HOME + "/.config" ) }/obsidian` let { vaults } = require( `${ appdata }/obsidian.json` ) // if the vaults list is empty/null, create it if( !vaults ) vaults = {} // add the vault entry vaults[ vault_id ] = { "path" : process.cwd(), "ts" : new Date().getTime() // 13-character timestamp - no idea if this even required // recommend leaving out the open statement. Using the Obsidian URI is going to prove more useful later anyway... } // write the update to the vault tracker fs.writeFileSync( `${ appdata }/obsidian.json`, JSON.stringify({ "vaults" : vaults })) ``` ## (Wait There's More!) The Window Cache Now, you would think that would be enough to get Obsidian URI's going, but it's not. (For reasons unknown to me) there also must be another `.json` file in that same directory to get it working, and that `.json` file is responsible for the Vault's Window's Cache. This file stores all of the settings used to launch the Vault's Window. That json file is promptly named `[vault_id].json` where `[vault_id]` is the vault ID you created above. In this case it is: `"ImA20CharacterString"` ### Node.js Snippet: ```javascript // imagine the above script here, lol (I'm lazy) ... fs.writeFileSync( `${ appdata }/${ vault_id }.json`, JSON.stringify({ "x": 0, // electron coordinates for where to place the window "y": 0, "width" : 600, "height" : 800, "isMaximized" : true, "devTools" : true, // yes, you can enable devtools to be launched immediately on open - super handy for testing "zoom" : 0 })) ``` ## Plugin Handling Now, at this point you can go ahead call the `"obsidian://open?path=[...]"` URI using your preferred method, but you may notice a few annoying things. Firstly that pesky "Trust Authors" pop-up. I'm sure your internal dialog as a plugin developer goes like this: *"Oh I've only trusted this vault the last 40,000 times I've cloned and opened it. I'm so glad that you've asked me again, just because I deleted the old one and cloned it back..."* ## The LevelDB's > [!danger] Warning: *LevelDBs DO NOT like you* > They do not want you to be inside them more than once > They like to break > Their developers don't like to document much on them, so good luck troubleshooting <sub>/sarcasm</sub> With that little warning out of the way, let me expand on it a bit. LevelDBs use Lockfiles to ensure that only one process is accessing them at a time. LevelDB databases are also not built to be intelligent. They are built to be efficient. One of the cost tradeoffs that Google went with when designing these is that they did not design a way to detect when a process was done with it, so... That leads to Lockfile issues. If a process doesn't tell the LevelDB to close before the process exits, it may leave the Lockfile locked. In fact this is the case for Node.js's "level" module. If you find that the lockfile is locked try killing the process that locked it. Then try deleting it. The LevelDBs in question and their lockfiles can be found in the following directories (the directory itself is the LevelDB): ```javascript `${ appdata }/Local Storage/leveldb/` `${ appdata }/Session Storage/leveldb/` // currently, obsidian doesn't really use this one all that much, but its there. ``` ### The `enable-plugins` flag The only use of the LevelDBs is to set the `enable-plugins` flag which is the flag used to "Trust Authors" Now, the flag has a weird format - I'm sure this has something to do with how Chrome or Electron works, but I'm still not sure yet: ```javascript `_app://obsidian.md\x00\x01${ key }` : `\x01${ value }` ``` where `key` is: - `enable-plugins` flag: - `` `enable-plugin-${ vault_id }` `` - `file-explorer-unfold` key: - *(not sure how this key is used, but its on all vaults)* - `` `${ vault_id }-file-explorer-unfold` `` and `value` is: - `enable-plugins` flag : `"true" //or "false"` - `file-explorer-unfold` key: `'["/"]'` ### Node.js Snippet: ```javascript const Level = require( "level" ).Level ... const Local : new Level( `${ appdata }/Local Storage/leveldb` ), // const Session : new Level( `${ appdata }/Session Storage/leveldb` ) //this is currently unused by Obsidian. It is empty. await Local.open() // use promisification, if you can't use the await kword // await Session.open() let keyprefix = "_app://obsidian.md\x00\x01" let keys = { "enable-plugin" : `enable-plugin-${ vault_id }`, "file-explorer-unfold" : `${ vault_id }-file-explorer-unfold` } let valueprefix = "\x01" let values = { "enable-plugin" : "true", "file-explorer-unfold" : '["/"]' } Object.keys( keys ).forEach( key => { let value = `${ valueprefix }${ values[ key ] }` key = `${ keyprefix }${ keys[ key ] }` Local.put( key, value ) }) await Local.close() // close the database! you'll regret it if you don't! // await Session.close() ``` ## The Community Plugins Tracker This one is even more straight forward than the Vault tracker. Mostly, because this file doesn't live internally, it lives in the`.obsidian` folder, and it is just a JSON array of all the Plugin ids that are to be enabled ### Node.js Snippet: ```javascript // let plugin_manifest = require( `path/to/plugin/manifest.json` ) let { plugins } = require( `path/to/test/vault/root/.obsidian/community-plugins.json` ) let plugin_id = "id" // || plugin_manifest.id // if the plugins list is empty/null, create it if( !plugins ) plugins = [] plugins.push( plugin_id ) fs.writeFileSync( ".obsidian/community-plugins.json", JSON.stringify( plugins )) ``` # Last Step: Launch the Obsidian URI using your preferred method! - I recommend node's `open` module, if you are going to use Node.js for this step