Monday, October 10, 2022

Triple boot iMac with Snow Leopard, El Capitan, and Batocera... Notes, Tips, Tricks, and Hints

Well, well, well.   I think I got it?  Use my notes at your own risk.  BACK UP BEFORE PARITIONING AND ERASE HD DATA!

Here's the basics for creating a triple boot Batocera on Intel Mac...  probably most iMacs, Macs, MacMini's, MacBooks, etc.

1. Find a wired keyboard.  If a windows one, remember "Windows" = "Apple/Command".
2. Create/validate 4 partitions... 
- A Rosetta capable MacOSX like Snow Leopard. 
- The lastest MacOS that runs on your hardware (it was El Capitan for me).
- A MSDOS-Fat32 partition of about 5-10 GB that is named "BATOCERA"
- A ExFAT partition named "Userdata".

3. Download Batocera for Intel Macs here https://batocera.org/download.
- Expand and open the .img.
- Copy everything from that image to your BATOCERA parition.
- (BalenEtcher is not required.)

4. Install rEFInd (you might need to use your Recovery Partition or a USB boot disc to get to Terminal to install if SIP is enabled.)  https://www.rodsbooks.com/refind/

5. Verify in Files that it found the Userdata partition, and then use the EmulationStation Developer menu item to run "Format" on the "Internal" drive to EXT4 format.  Reboot.

6. Add roms and have fun!

That's it.  The wordy section with tips and pitfalls follows.  Thanks


----------------wordy section------------------


I saved an iMac with 3GB of memory and a 320 GB drive and a dual core Intel processor from the dump.  A friend got it for $10 and couldn't figure out how to fix and flip.  I got into it, made a boot USB for the OS's I wanted, and got it all repaired and fixed.  (The max Rosetta/MacOS9 compatible OS is Snow Leopard, so that's one 120 GB partition, The max OS for this OS is El Capitan, so that's next at another 120 GB, then I have a 10 GB FAT partition for Batocera, and then finally 70 left for the Userdata partition.

I didn't find a lot of information on how to set up Batocera on a dual boot Mac from google, and so I thought I'd document what worked.  (Note: Some message boards forward you to this page... https://wiki.batocera.org/dual_boot_ubuntu_batocera.linux  But, that doesn't have ANY information for MacOS and is written like this dual boot stuff is all horribly impossible and ill advised.  But, with what I did below, well...  it ended up working just fine.)

As with a lot of these dumps, this is not going to be highly formatted or pretty.  I just wanted to get it all down before I went on to the next thing.

1. Windows keyboards on Mac...  The windows key is the command/"four circles weird icon"/Apple key.  Also, a wireless Logitech doesn't always work for the Command-R (to start in recovery mode) or the very important Alt (to start in boot picker) methods of holding those as the computer restarts.  So, having a wired keyboard during this is important.  (Once you are booted to the mode you want, everything is fine...)

2. Getting the partitions right.  Ugh.  (You'll probably want to put windows on your machine.  I just didn't do that.   But shouldn't have any issue. But, it's a really old mac and I have lots of windows machines.)  So, I was focused on having a solid Rosetta machine again to run old stuff.  I always regretted updating past that.  Regardless, you can do as many partitions as you want.  Except for that the partition manager is really weird and wonky on the Mac DiskUtility.  

What I finally figured out through all of this was that the way to "fix" issues where the partitions won't get set, is to just take the last MacOSJournaled partition and have it reoccupy all the remaining open space by deleting all the partitions that aren't just working right.  

Once you've done that... it's actually simple.  

2. a. Just select the last MacOSJournaled partition that is too big and has free-space, and hit "+" and then "+" again.  Then resize those as you want.  And on the 2 new partitions... I HAD to partition them as MacOSJournaled FIRST.  Every time I tried to go straight to the formats I wanted... then it failed.

2. b. But if you get the partitions right with MacOSJournaled _THEN_ it's time to change them and rename them.  So, call the first tiny one BATOCERA and formatted it as FAT.  (I made it 10GB to prevent against future expansions).  

2. c. And then I called the bigger one "Userdata" and formatted it as EXFAT (for now... we'll fix that to EXT4 cause FuturePinball - https://wiki.batocera.org/systems:fpinball says it prefers that, even though I assume this computer is too slow for that.)

So, now, you should have all your "legit" partitions for MacOS versions and Windows and Linux versions... and then for me... 

To summarize step 2... I made my last 2 partitions... "BATOCERA" (MS-DOS FAT) and "Userdata" (ExFAT)...  Which is how ye old USB partitions look when you make a USB drive.

(NOTE: This method _really_ scared me and confused me.  I thought I could just install this on BATOCERA as a BIG ASS FAT32 DRIVE cause that's what mounted on the USB stick I'd made at first... But one big FAT32 drive had no space when I did ROM transfer... but that's because of how "Userdata" works... it's a separate partition with lots of space that is leveraged by the BATOCERA Linux boot.

I found one thread that clued me into this all here: https://wiki.batocera.org/batocera.linux_architecture  And it very clearly explains that you need 2 partitions!  I should have had that _duh_ moment, but I didn't because I'd read that horrible link I crossed through up at the start.  I thought somehow that "Userdata" was materialized by magic by the first launch.  I thought it would maybe create a rogue Userdata that took over my good partitions.  I was scared.

I spend at lot of time switching between "External" and "Internal" and the various drives that EmulationStation offered.... But, what _I_ wanted was MacOS installed on my harddrive.  No USB or external or network stuff.  Just _there_ whenever I wanted to boot in.  So, 2 partitions is the trick.  (Spoiler, first boot with "Userdata" as a partition existing... IT FOUND IT AUTOMATICALLY when "internal", the default, is set in Emulation Station.))

Where was I.  Ok, so, I have my partitions.

3. Next step, go to https://batocera.org/download and get the MacOS Intel version.

3. a. (NOTE: Do they want you to use BalenEtcher to install and all that in the documentation???  yeah, they do.  But do you need to???  Not really.   Not really at all.  Why???  BECAUSE... Emulation Station has a very scary formatting utility built in.  Ack!  So, you don't actually need to do anything but dump the Image's BATOCERA data into the BATOCERA partition.  That's it.)

3. b.  Just open the .img, and you'll get _only_ BATOCERA... Just copy all the files (batocera-boot.conf, and the folders boot, EFI, and tools) over to your partition... also named BATOCERA.

3. c. But but but... they said to use BalenEtcher.  I know.  But, you don't need to.

4. At this point, you will probably have gotten here because you're struggle and have had to hold "ALT" about a million times to play with things.  Here's part of what makes all this weird... most of the guides and message board posts whine about not being able to have a graceful autoboot menu by default.  That's because they haven't messed with rEFInd.  (https://wiki.batocera.org/dual_boot_ubuntu_batocera.linux, the x'd out link above, does mention rEFInd, but it's under an "iOS" section and makes it sound like you'd better hope and pray it finds your partition.  Poppycock.  Not sure who wrote that, but it clued me into reading about the utility, and it's great.)

4. a. rEFInd... Set up the boot menu with an amazing boot utility that replaces the "hold ALT during reboot" with a cool booting utility by Roderick W. Smith.  https://www.rodsbooks.com/refind/  Go ahead and get that.  The documentation is wonderfully verbose and accurate.  I had to use the SIP workaround in bash at the terminal after booting from an install USB because my recovery partition wouldn't boot (likely because this dang computer had a hell of a hard life and had something go corrupt on the original owner in the boot sequence - They messed up something somehow... anyway...).  If you can be methodical and follow directions, it is doable.  I highly recommend it so you can have your Mac boot the way you want.

5. So, now it's time to boot up BATOCERA with rEFInd.  Once you're in, it's too loud (hope you're keyboard has volume controls on it), and then it's too loud again.  You'll wake up the wife.  

5. a.  It will take a long time your first time, as I assume it's going ahead and creating the default files and folders on Userdata.  (You will be destroying them in a second.)

5. b. Once you have the interface, hit the F1 key and go to the file browser.  Make sure that when you go to Roms or Bios or Userdata, that the free size on all of them is what you made that partition.  (In my case, it was the 70 GiB partition.)

5. c. For sure for sure make sure that's right.  Cause, the next step is for you to delete it and put EXT4 on it!  So, the way you do this is to quit Files, and then hit space bar, and then go into the Menu.  As documented by the amazing link here: https://wiki.batocera.org/batocera.linux_architecture , you can then follow it's advice to go to Menu -> System Settings -> Frontend Developer -> Format. 

5. d. Ugh... Formatting is scary.  Scary scary.  But, remember... You made your partitions.  You defaulted to "Internal" as the location.  And, most importantly, you did step 5. b. and verified that the "Internal" partition it found was indeed Userdata because the size of the drive matched the size of your partition.

5. e. You are making regular backups of your main system, right?  And you did before you started partitioning everything, right?  Right?

5. f. So, go ahead and format it to EXT4.  And ta-da, it happens so fast, you barely realize it.

5. g. This is where I point out that it seems like _ANY_ EXT4 capable partitioning software will work.  But, most guides need all sorts of Ubuntu USB or MacFUSE installs or what have you...  And this built in functionality works great after you get over being scared.

6. So, that's pretty much it.  Reboot into batocera, and see that, now, tada, you now have an EXT4 userdata drive.  The MacOS won't love it (unless maybe if you use MacFUSE), but who cares.  Moving ROMS is super simple using the myriad of guides for either using \\BATOCERA or using FTP or using SMB or what have you.  The only downside of course is that you'll need an extra computer or laptop to interface...  This is where an Ubuntu boot USB or MacFUSE might help... and also... I haven't tried mapping the other drives as I'm not a linux-y kind of guy.  If that is you, it might work.

6. a. Note: YOU GOTTA TELL EMULATIONSTATION TO HOOK UP TO YOUR WIFI NETWORK MANUALLY THROUGH THE MENU all the time.... Ugh...


Anyway, I hope that long ass wordy guide helped someone struggling/scared with making a dual boot Mac that runs Batocera.


Good Luck!

Saturday, August 6, 2022

Notes on Rebuilding Title.db on my 3DS after 3hs hshop loss of battery and out of space on SD Card using save3ds and rebuild-title-database

Huge thank yous to all the 3DS homebrew developers including ihaveamac, wwylele, Ryuzaki_MrL, all the MacOS rust and brew and python devs, and to ZeroSkill for publishing their code and doing all the real work.  Hopefully this guide can help someone use their amazing tools to recover their 3DS titles after an SD card corruption.

So, I found out there is the amazing hshop homebrew https://hshop.erista.me/3hs for 3DS installation of games on a luma / godmode9 hacked 3DS via https://3ds.hacks.guide

I went nuts on it like a kid in a candy shop, and twice, corrupted my 3DS's title.db!

The symptoms of a corrupted title.db is that everything but DSiware titles and native 3DS utilities disappear from your 3DS!  Ack!  Panic!  All my icons are gone!  SD card is fine...  hmmm...  everything still in all the folders.  Did Nintendo find out I've been naughty and zap me?  I was in a panic.

But, I hopped on the 3hs discord support forum, and ZeroSkill quickly pointed me in the right direction.

Documenting this here for future folks as the reality of a title.db recovery is that it's pretty convoluted and extensive.  So I'm putting the notes I remember here.  This won't be definitive as a guide.  I'm writing it wordy and horrible as my brain remembers.  So, my apologies for lack of editing and clarity.

But hopefully it helps someone in need get there faster and with less pain.

Before you begin...

So, there are two main utilities you will need.  

https://github.com/ihaveamac/rebuild-title-database

https://github.com/wwylele/save3ds

These utilities 100% worked for me, but they are _not_ set up as easy to use applications yet.  You will need Linux or MacOSX to use Save3ds.  Yes.  I put that in the worst font possible.  I tried to find some geocities flash code even, but alas...  Don't get halfway through this on a windows machine, only to realize you're screwed halfway through.  You need to be comfortable with command lines, or willing to learn.  And you might need to find a way to run Linux if you don't have a Mac available.

Again, Save3ds requires Linux or MacOSX.  And it requires use of command lines.  If you don't have those available... well...

As for me, I started on Windows just doing the first part of rebuild-title-database, only to get to save3ds, and be lucky that I am in a house with Windows, Mac, and Linux machines.

So, not only will you need those 2, but you'll need to be able to compile and run them... which takes a ton of stuff too, like Python and compiling things like rust on Mac's Terminal.  I'm in that zone of knowing enough to figure it out, but being scared and confused the whole way.

BUT I DID IT!  lol

So, first... I was on Windows 7 and I got rebuild-title-database to work finally.  Then, I switched to my MacBook to finish with Save3ds.  (If you are using Linux, hopefully you are smarter than me and will laugh mockingly at my commandline skills.)

Steps (27)!

1. Before you start, you have to follow this guide:  https://ihaveamac.github.io/dump.html Turns out for me, the way to get it to work was to launch my 2DSXL without the SD card in it, and then after hitting save on the default luma screen, it opens straight into GodMode9 / GM9.  This was important because when your title.db on your SD card dies, you don't have the icon anymore to open God Mode, and the methods on the dump page weren't working for me.

NOTE: It was easiest for me in the command line, to mount the SD card on my pc and then drag the boot9.bin and moveable.sed files to the same folder where I unzipped rebuild-title-database.  You can leave them on the SD card in the out folder and reference them there.  But, it's a lot of typing.

2. Then, you go back, and you realize, for me at least, that Python wasn't installed.  So, I had to install Python.  

Quick note, if you're on Windows 7 like I am on my game/mame/virtual pin pc... you need to scroll down through lots of versions of Python before you find one that is compatible with Windows 7.  Took me a bit to figure that out.

Another quick note, when you install python, you have to check "add Python to your path".  If you don't, then the command line doesn't work.  I had to uninstall and reinstall to recheck the box.

3. Once python is installed, you go to your terminal, and you change the location you're at to the location you unzipped your download of rebuild-title-database.  For me, I had unzipped to me F: drive.  Turns out, in the examples, they use "F:" to be the location of SD card...  That confused me.  Ope!

<py3-cmd> rebuild-title-database.py -b boot9.bin -m movable.sed -s F: -o out

This is what is on the readme page.  So, mine actually ended up looking like this...

python rebuild-title-database.py -b boot9.bin -m movable.sed -s M: -o out

This is because 1. I moved boot9.bin and movable.sed to the folder that contains rebuild-title-database.py.  And, because I changed from c:\whatever\ to the folder I actually unzipped, which was f:.   So, on the terminal, it looked like 

F:\> python rebuild-title-database.py -b boot9.bin -m movable.sed -s M: -0 out

I'm pretty sure, if M: is your SD card, and you didn't move boot9.bin, you could have put M:\gm9\out\boot9.bin in that line...

F:\> python rebuild-title-database.py -b M:\gm9\out\boot9.bin -m M:\gm9\out\movable.sed -s M: -0 out

NOTE: At some point, I believe I also needed to get seeddb.bin and put it in the same folder as the rest of rebuild-title-database.  You can find that here:  https://github.com/ihaveamac/3DS-rom-tools/wiki/SeedDB-list

4. Ok, so, when you run rebuild-title-database after a corruption like mine, what happens, is it starts parsing out the files in your ID0/ID1 folder structure...

5. And it saves those file ID's out to the "out" folder... Which, in my example, was F:\out\.  Because we only defined the word "out" in that statement.  (I assume, again, you could change the word "out" in the statement, to _any_ windows path.

6. BUT, when I'd run this, it would Fail.  Over and over.  I couldn't figure out why.  But, finally, I figured it out.  The failure occurs at the file when hshop died.  The first time, hshop failed because the battery died mid download.  The second time, hshop failed because I ran out of space on the SD card.  The trick is to read the first 8 and the last 8 digits of the file ID near the end of the error code, and then go find those folders, and delete the final folder from the SD card.

The ID will look like 0004000e001d1300 in the error message.

The path on your SD card will look like this...  

\Nintendo 3DS\dbxxxxxxID0xxxxxxxxx14\f6axxxxxxxxID1xxxxxxxx44d\title\0004000e\001d1300\

You will want to delete this Folder: \001d1300\

7. Once you delete the offending folder, you will lose the partially downloaded/corrupt game.  But, it will free everything else back up for when you rerun rebuild-title-database again.

8. When you are successfully, you will now have a folder called "out" with a ton of files that look like "0004000e001d1300".  This is all the installed titles on your 3DS's SD card, minus the one you just deleted.

IMPORTANT:  Copy this "out" folder to your SD Card!  Because you need the folder on your Mac or Unix system for the next step...  I renamed the folder to "titleexport" for myself to keep track of it because "out" is used a lot by other apps on the 3DS card.  

9. So, NOW, with your titles exported by rebuild-title-database, you'll need to use https://github.com/wwylele/save3ds to rebuild and reseed the title.db.  (I also did this to import.db also, see later.)

10. So, Save3Ds isn't hard to use, but it's a pain to install if you're not a programmer.  I did _not_ write down everything I went through when I installed this, but needless to say, you have to go through a bunch of help files to get everything set up.

11. Installing Rust is required.  Good luck.  I forget everything I did to get it going, but somehow I did.  https://www.rust-lang.org/tools/install looks right, but I think I actually had to use this...  https://ports.macports.org/port/cargo/  with the help file from here... https://doc.rust-lang.org/cargo/getting-started/installation.html

12. Then, the command for mac is to "brew" a "cask".  It seems that "cask" is used differently now, and "brew" is something you have to install yourself..

brew cask install osxfuse && brew install pkg-config

13. So, you can get Brew through here...  https://brew.sh/

14. But, then, it didn't run, and so I think I had to change where "cask" was, and possibly had to do the 2 installs separately.  Again, I was in pure "get this done" mode, and wasn't thinking about documenting.  My browser history has this link, so I assume I was able to get brew working with one of the commands here...  OR, I found the install location and ran it from that.  https://apple.stackexchange.com/questions/148901/why-does-my-brew-installation-not-work

15. Anyway, somehow, I got it installed!  Yay!

16. And then, I tried to run the line...  And oh my, if you're not used to command lines for apps that do a million things, hold on to your seat!  https://github.com/wwylele/save3ds has really good documentation, but it's still super confusing on a Mac to make work because of MacOSX's quirks and because, as a super useful tool, save3ds has a ton of options for a multitude of uses beyond just this task.

17.  First tip, my SD card was called by my Mac - NO NAME.  That space in "NO NAME" really messed me up.  So, I had to title my SD Card.  I used the wonderful fix of ... "NONAME".

18. Next, much like needing to be in the proper folder where the .py file is when running the python script, you need to spend time figuring out where to point things and locate things on the Mac.  One trick... If you find a folder you like, and you don't know the path, you can drag the folder into terminal, and it writes the path for you.  That REALLY helps when you are a newb.

19. Cut to the chase, here's the lines that worked for me on the terminal...  And, I'll explain what they are in subsequent steps...

/Users/mbpuser/save3ds-master/target/debug/save3ds_fuse --db sdtitle /Users/mbpuser/save3ds-master/target/titleexport --sd /Volumes/NONAME --movable /Volumes/NONAME/gm9/out/movable.sed --boot9 /Volumes/NONAME/gm9/out/boot9.bin --import


Users/mbpuser/save3ds-master/target/debug/save3ds_fuse --db sdimport /Users/mbpuser/save3ds-master/target/titleexport --sd /Volumes/NONAME --movable /Volumes/NONAME/gm9/out/movable.sed --boot9 /Volumes/NONAME/gm9/out/boot9.bin --import


/Users/mbpuser/save3ds-master/target/titleexport  

(All Titles exported by the python script into the "out" folder by rebuild-title-database on the Windows machine need to go here.  Other names would _certainly_ work, including "out".)


20.  So, the first thing is that save3ds worked best for me with everything simply held in the default locations of the save3ds install.  Part of my struggle was the "NO NAME" vs. "NONAME" problem.  So, I probably could have left files on the SD card but as I was fixing errors, placing the files in the same folder as the install got me past those issues.

Here's what each part of my command line statements means...
/Users/mbpuser/save3ds-master/target/debug/save3ds_fuse    <-This is where my install of save3ds was located.
--db sdtitle  <- This tells save3ds that it is a database action and that it's rebuilding sdtitle ON YOUR SD CARD
/Users/mbpuser/save3ds-master/target/titleexport    <- This is the folder where all of the rebuild-title-database files in the "out" folder were stored by me.  You should set it to the folder where your output is.  I assume it could be on the SD card in a folder named "out" for most people.
--sd /Volumes/NONAME   <- This is the path to the root of my SD card.  
--movable /Volumes/NONAME/gm9/out/movable.sed   <- This is the path on my SD card to the movable.sed file that GodMode9 exported back in step 1.
--boot9 /Volumes/NONAME/gm9/out/boot9.bin  <- This is the path on my SD card to the boot9.bin file that GodMode9 exported back in step 1.
--import   <- This tells save3ds that you are wanting to import the file to the proper place on your SD card.

On the second statement, the only thing different is "sdimport" instead of "sdtitle".  These are how the save3ds readme defines these:
  • sdtitle refers to the file SDMC:/Nintendo 3DS/<ID0>/<ID1>/dbs/title.db
  • sdimport refers to the file SDMC:/Nintendo 3DS/<ID0>/<ID1>/dbs/import.db
I honestly don't know if I needed to do import.db.  But, since it was in the same folder as title.db, I did it anyway.

21.   So, if you run the utility, and everything goes right, all that work just results in a title.db and import.db of almost the same size with a new date and time stamp.

22.  Excitedly, you run, put your SD card into the slot, and...  NOTHING.  Nothing happens like a fix.  You boot the 3DS, and it looks like it all failed and nothing new has happened.

23.  Cry, a little, panic.  Restart a couple times, put the card in and out.  IDK  what I did, but both times, after a few cycles of this, all the sudden... poof!  Hundreds of new install gift boxes appear.  Yay!

24.  *sigh*  So, you do lose all your nice folders and structure for everything.  But, at least all your apps are back.  

Note: You also lose all your badge placements in apps.  (Folder badges seem to stay in place.)  But, this definitely messes up using GYTB because it can't find anything and errors out.

25.  But first, unwrapping all those is a pain in the ass.  Fortunately, you can use chthulu to unwrap them all.  https://gbatemp.net/threads/release-cthulhu-app-for-managing-play-time-step-history-and-cached-icon-data.439485/  It's available in Universal Updater.

26. Since your badges get messed up, you have to rebuild them.  So, you can redo your badges by going into data management, deleting the badge arcade's extra data, and then restoring it in the badge arcade as guided by the app.  https://www.reddit.com/r/3DS/comments/3te5at/nintendo_badge_arcade_error/ is one example of a thread on this.

27.  So, that's pretty much it.  Again, my thanks to the entire community of folks who make these tools. I'm always in awe of the generosity of those who make their code opensource and release for people like me to use.  Thank you thank you thank you.

Triple boot iMac with Snow Leopard, El Capitan, and Batocera... Notes, Tips, Tricks, and Hints

Well, well, well.   I think I got it?  Use my notes at your own risk.  BACK UP BEFORE PARITIONING AND ERASE HD DATA! Here's the basics f...