Notes on Rebuilding Title.db on my 3DS after 3hs hshop loss of battery and out of space on SD Card using save3ds and rebuild-title-database

Huge thank yous to all the 3DS homebrew developers including ihaveamac, wwylele, Ryuzaki_MrL, all the MacOS rust and brew and python devs, and to ZeroSkill for publishing their code and doing all the real work.  Hopefully this guide can help someone use their amazing tools to recover their 3DS titles after an SD card corruption.

So, I found out there is the amazing hshop homebrew https://hshop.erista.me/3hs for 3DS installation of games on a luma / godmode9 hacked 3DS via https://3ds.hacks.guide. 

I went nuts on it like a kid in a candy shop, and twice, corrupted my 3DS's title.db!

The symptoms of a corrupted title.db is that everything but DSiware titles and native 3DS utilities disappear from your 3DS!  Ack!  Panic!  All my icons are gone!  SD card is fine...  hmmm...  everything still in all the folders.  Did Nintendo find out I've been naughty and zap me?  I was in a panic.

But, I hopped on the 3hs discord support forum, and ZeroSkill quickly pointed me in the right direction.

Documenting this here for future folks as the reality of a title.db recovery is that it's pretty convoluted and extensive.  So I'm putting the notes I remember here.  This won't be definitive as a guide.  I'm writing it wordy and horrible as my brain remembers.  So, my apologies for lack of editing and clarity.

But hopefully it helps someone in need get there faster and with less pain.

Before you begin...

So, there are two main utilities you will need.  

https://github.com/ihaveamac/rebuild-title-database

https://github.com/wwylele/save3ds

These utilities 100% worked for me, but they are _not_ set up as easy to use applications yet.  You will need Linux or MacOSX to use Save3ds.  Yes.  I put that in the worst font possible.  I tried to find some geocities flash code even, but alas...  Don't get halfway through this on a windows machine, only to realize you're screwed halfway through.  You need to be comfortable with command lines, or willing to learn.  And you might need to find a way to run Linux if you don't have a Mac available.

Again, Save3ds requires Linux or MacOSX.  And it requires use of command lines.  If you don't have those available... well...

As for me, I started on Windows just doing the first part of rebuild-title-database, only to get to save3ds, and be lucky that I am in a house with Windows, Mac, and Linux machines.

So, not only will you need those 2, but you'll need to be able to compile and run them... which takes a ton of stuff too, like Python and compiling things like rust on Mac's Terminal.  I'm in that zone of knowing enough to figure it out, but being scared and confused the whole way.

BUT I DID IT!  lol

So, first... I was on Windows 7 and I got rebuild-title-database to work finally.  Then, I switched to my MacBook to finish with Save3ds.  (If you are using Linux, hopefully you are smarter than me and will laugh mockingly at my commandline skills.)

Steps (27)!

1. Before you start, you have to follow this guide:  https://ihaveamac.github.io/dump.html Turns out for me, the way to get it to work was to launch my 2DSXL without the SD card in it, and then after hitting save on the default luma screen, it opens straight into GodMode9 / GM9.  This was important because when your title.db on your SD card dies, you don't have the icon anymore to open God Mode, and the methods on the dump page weren't working for me.

NOTE: It was easiest for me in the command line, to mount the SD card on my pc and then drag the boot9.bin and moveable.sed files to the same folder where I unzipped rebuild-title-database.  You can leave them on the SD card in the out folder and reference them there.  But, it's a lot of typing.

2. Then, you go back, and you realize, for me at least, that Python wasn't installed.  So, I had to install Python.  

Quick note, if you're on Windows 7 like I am on my game/mame/virtual pin pc... you need to scroll down through lots of versions of Python before you find one that is compatible with Windows 7.  Took me a bit to figure that out.

Another quick note, when you install python, you have to check "add Python to your path".  If you don't, then the command line doesn't work.  I had to uninstall and reinstall to recheck the box.

3. Once python is installed, you go to your terminal, and you change the location you're at to the location you unzipped your download of rebuild-title-database.  For me, I had unzipped to me F: drive.  Turns out, in the examples, they use "F:" to be the location of SD card...  That confused me.  Ope!

<py3-cmd> rebuild-title-database.py -b boot9.bin -m movable.sed -s F: -o out

This is what is on the readme page.  So, mine actually ended up looking like this...

python rebuild-title-database.py -b boot9.bin -m movable.sed -s M: -o out

This is because 1. I moved boot9.bin and movable.sed to the folder that contains rebuild-title-database.py.  And, because I changed from c:\whatever\ to the folder I actually unzipped, which was f:.   So, on the terminal, it looked like 

F:\> python rebuild-title-database.py -b boot9.bin -m movable.sed -s M: -0 out

I'm pretty sure, if M: is your SD card, and you didn't move boot9.bin, you could have put M:\gm9\out\boot9.bin in that line...

F:\> python rebuild-title-database.py -b M:\gm9\out\boot9.bin -m M:\gm9\out\movable.sed -s M: -0 out

NOTE: At some point, I believe I also needed to get seeddb.bin and put it in the same folder as the rest of rebuild-title-database.  You can find that here:  https://github.com/ihaveamac/3DS-rom-tools/wiki/SeedDB-list

4. Ok, so, when you run rebuild-title-database after a corruption like mine, what happens, is it starts parsing out the files in your ID0/ID1 folder structure...

5. And it saves those file ID's out to the "out" folder... Which, in my example, was F:\out\.  Because we only defined the word "out" in that statement.  (I assume, again, you could change the word "out" in the statement, to _any_ windows path.

6. BUT, when I'd run this, it would Fail.  Over and over.  I couldn't figure out why.  But, finally, I figured it out.  The failure occurs at the file when hshop died.  The first time, hshop failed because the battery died mid download.  The second time, hshop failed because I ran out of space on the SD card.  The trick is to read the first 8 and the last 8 digits of the file ID near the end of the error code, and then go find those folders, and delete the final folder from the SD card.

The ID will look like 0004000e001d1300 in the error message.

The path on your SD card will look like this...  

\Nintendo 3DS\dbxxxxxxID0xxxxxxxxx14\f6axxxxxxxxID1xxxxxxxx44d\title\0004000e\001d1300\

You will want to delete this Folder: \001d1300\

7. Once you delete the offending folder, you will lose the partially downloaded/corrupt game.  But, it will free everything else back up for when you rerun rebuild-title-database again.

8. When you are successfully, you will now have a folder called "out" with a ton of files that look like "0004000e001d1300".  This is all the installed titles on your 3DS's SD card, minus the one you just deleted.

IMPORTANT:  Copy this "out" folder to your SD Card!  Because you need the folder on your Mac or Unix system for the next step...  I renamed the folder to "titleexport" for myself to keep track of it because "out" is used a lot by other apps on the 3DS card.  

9. So, NOW, with your titles exported by rebuild-title-database, you'll need to use https://github.com/wwylele/save3ds to rebuild and reseed the title.db.  (I also did this to import.db also, see later.)

10. So, Save3Ds isn't hard to use, but it's a pain to install if you're not a programmer.  I did _not_ write down everything I went through when I installed this, but needless to say, you have to go through a bunch of help files to get everything set up.

11. Installing Rust is required.  Good luck.  I forget everything I did to get it going, but somehow I did.  https://www.rust-lang.org/tools/install looks right, but I think I actually had to use this...  https://ports.macports.org/port/cargo/  with the help file from here... https://doc.rust-lang.org/cargo/getting-started/installation.html

12. Then, the command for mac is to "brew" a "cask".  It seems that "cask" is used differently now, and "brew" is something you have to install yourself..

brew cask install osxfuse && brew install pkg-config

13. So, you can get Brew through here...  https://brew.sh/

14. But, then, it didn't run, and so I think I had to change where "cask" was, and possibly had to do the 2 installs separately.  Again, I was in pure "get this done" mode, and wasn't thinking about documenting.  My browser history has this link, so I assume I was able to get brew working with one of the commands here...  OR, I found the install location and ran it from that.  https://apple.stackexchange.com/questions/148901/why-does-my-brew-installation-not-work

15. Anyway, somehow, I got it installed!  Yay!

16. And then, I tried to run the line...  And oh my, if you're not used to command lines for apps that do a million things, hold on to your seat!  https://github.com/wwylele/save3ds has really good documentation, but it's still super confusing on a Mac to make work because of MacOSX's quirks and because, as a super useful tool, save3ds has a ton of options for a multitude of uses beyond just this task.

17.  First tip, my SD card was called by my Mac - NO NAME.  That space in "NO NAME" really messed me up.  So, I had to title my SD Card.  I used the wonderful fix of ... "NONAME".

18. Next, much like needing to be in the proper folder where the .py file is when running the python script, you need to spend time figuring out where to point things and locate things on the Mac.  One trick... If you find a folder you like, and you don't know the path, you can drag the folder into terminal, and it writes the path for you.  That REALLY helps when you are a newb.

19. Cut to the chase, here's the lines that worked for me on the terminal...  And, I'll explain what they are in subsequent steps...

/Users/mbpuser/save3ds-master/target/debug/save3ds_fuse --db sdtitle /Users/mbpuser/save3ds-master/target/titleexport --sd /Volumes/NONAME --movable /Volumes/NONAME/gm9/out/movable.sed --boot9 /Volumes/NONAME/gm9/out/boot9.bin --import

Users/mbpuser/save3ds-master/target/debug/save3ds_fuse --db sdimport /Users/mbpuser/save3ds-master/target/titleexport --sd /Volumes/NONAME --movable /Volumes/NONAME/gm9/out/movable.sed --boot9 /Volumes/NONAME/gm9/out/boot9.bin --import

/Users/mbpuser/save3ds-master/target/titleexport  

(All Titles exported by the python script into the "out" folder by rebuild-title-database on the Windows machine need to go here.  Other names would _certainly_ work, including "out".)

20.  So, the first thing is that save3ds worked best for me with everything simply held in the default locations of the save3ds install.  Part of my struggle was the "NO NAME" vs. "NONAME" problem.  So, I probably could have left files on the SD card but as I was fixing errors, placing the files in the same folder as the install got me past those issues.

Here's what each part of my command line statements means...

/Users/mbpuser/save3ds-master/target/debug/save3ds_fuse    <-This is where my install of save3ds was located.

--db sdtitle  <- This tells save3ds that it is a database action and that it's rebuilding sdtitle ON YOUR SD CARD

/Users/mbpuser/save3ds-master/target/titleexport    <- This is the folder where all of the rebuild-title-database files in the "out" folder were stored by me.  You should set it to the folder where your output is.  I assume it could be on the SD card in a folder named "out" for most people.

--sd /Volumes/NONAME   <- This is the path to the root of my SD card.  

--movable /Volumes/NONAME/gm9/out/movable.sed   <- This is the path on my SD card to the movable.sed file that GodMode9 exported back in step 1.

--boot9 /Volumes/NONAME/gm9/out/boot9.bin  <- This is the path on my SD card to the boot9.bin file that GodMode9 exported back in step 1.

--import   <- This tells save3ds that you are wanting to import the file to the proper place on your SD card.

On the second statement, the only thing different is "sdimport" instead of "sdtitle".  These are how the save3ds readme defines these:

• sdtitle refers to the file SDMC:/Nintendo 3DS/<ID0>/<ID1>/dbs/title.db
• sdimport refers to the file SDMC:/Nintendo 3DS/<ID0>/<ID1>/dbs/import.db

I honestly don't know if I needed to do import.db.  But, since it was in the same folder as title.db, I did it anyway.

21.   So, if you run the utility, and everything goes right, all that work just results in a title.db and import.db of almost the same size with a new date and time stamp.

22.  Excitedly, you run, put your SD card into the slot, and...  NOTHING.  Nothing happens like a fix.  You boot the 3DS, and it looks like it all failed and nothing new has happened.

23.  Cry, a little, panic.  Restart a couple times, put the card in and out.  IDK  what I did, but both times, after a few cycles of this, all the sudden... poof!  Hundreds of new install gift boxes appear.  Yay!

24.  *sigh*  So, you do lose all your nice folders and structure for everything.  But, at least all your apps are back.  

Note: You also lose all your badge placements in apps.  (Folder badges seem to stay in place.)  But, this definitely messes up using GYTB because it can't find anything and errors out.

25.  But first, unwrapping all those is a pain in the ass.  Fortunately, you can use chthulu to unwrap them all.  https://gbatemp.net/threads/release-cthulhu-app-for-managing-play-time-step-history-and-cached-icon-data.439485/  It's available in Universal Updater.

26. Since your badges get messed up, you have to rebuild them.  So, you can redo your badges by going into data management, deleting the badge arcade's extra data, and then restoring it in the badge arcade as guided by the app.  https://www.reddit.com/r/3DS/comments/3te5at/nintendo_badge_arcade_error/ is one example of a thread on this.

27.  So, that's pretty much it.  Again, my thanks to the entire community of folks who make these tools. I'm always in awe of the generosity of those who make their code opensource and release for people like me to use.  Thank you thank you thank you.