Cloning SD cards

Posted by: @pugwash

Clone the drive sda onto drive sdb:
$ dd if=/dev/sda of=/dev/sdb

This is nowhere as easy as it sounds.

To being with before I even plug in any SD cards my Linux computer already lists the following as supposedly already existing.  I have no clue what these are.

sda
sda1
sda2
sda3
sda4

When I plug in the Jeston Nano SD card the following appear:

sdb
sdb1
sdb2
sdb3
sdb4
...
sdb12

This is pretty much the same thing I got on the Windows machine.  It's like it's been broken up into 12 partitions.

Then when I plug in the blank SD card the following shows up

sdc
sdc1

I guess this is just showing me that this  SD card only has a single partition?

In any case I have no idea what the first sda that has four partitions is.  It's nothing I'm aware of.

The Jetson Nano SD card appears to be showing up as sdb, and the new empty card appears to be showing up as sdc.

So should I just use the following command instead?

$ dd if=/dev/sdb of=/dev/sdc

Instead of cloning from a to b I should clone from b to c? Right?

And will this then clone all 12 partitions onto the new SD card?

I suppose it can't hurt to try.

I have no clue what the sda with 4 partitions is on this computer.  This is a notebook computer.  Perhaps it already has some sort of internal backup ROM in it that it lists as an SD card?

It looks to me like I need to clone from sdb to sdc?

I sure hope this doesn't screw up my Linux computer!   I don't want to overwrite the hard drive.

Cloning the SD card on Windows:

I've already backed up Raspberry Pi System cards using Win32DiskImager so I've decided to try using the same method on the Jetson Nano Card.

There is a slight difference.   The Raspberry Pi SD cards only have two partitions.  One is the Boot partition and the other is an unnamed partition.   To back up the entire SD card all that is require is to copy the Boot partition to an image file.  This actually copies the entire  SD card anyway.

So now that I see how Linux is doing this I'm suspecting that Windows works in a very similar way.   So I'm giving that a try right now.  When I plug the SD card into Windows it shows up as 12 drives on my computer they have been named G,... thru,... R.    At first I didn't know what to do with that because I can only select on of them at a time with Win32DiskImager.   However now I'm thinking that I only need to select the first one (in my case that will be drive G). 

I then create an empty *.img file given whatever name I like, and save drive G to that image file.   When that's done I should be able to just write that file back out to a new SD card.   That new SD card will then have all 12 partitions written to it after this process is over.  Hopefully.   This is the way it works for the Raspberry Pi SD cards.  The only difference is the Raspberry Pi cards only have two partitions while the Jetson Nano has 12.

In any case I think this is going to work because it's still reading the SD card to the image file.  If this was only 1/12th of the SD card it should have been done by now.    So I suspect that it's copying the entire SD card.

When it's done I write it to the new SD card and see it works.  I'll report back later with the results. ? 

By the way, this is not an attempt to avoid Linux.   I need to know both systems.   So I'll probably do this on Linux too.   I have tons of pies and other SD cards that I like to back up.  So it's nice having all the options I can get.   I also imagine that I'll end up with a lot of additional system cards for the Jetson Nano as well.  Paul McWhorter is demanding that I keep one SD card just for his lessons.   I'll be glad to do that.  Similarly I have various experiments that I'll be doing as well, and I'll be keeping those experiments on different SD cards as well.   So I'll be making a lot of new system cards over the next few months.   May as well get this SD card cloning down pat right now. ? 

Just for the record Win32DiskImager just finished copying the SD card to an image file.  The image file is  65 GB so it got the whole thing.   I'll clone the other SD card later from that image file and report back again when I've done that, but based on how it looks right now I'm pretty confident that it copied the whole card.

I wrote the new SD card and just booted it up in the Jetson Nano.   Everything appears to be working just like the original card.   So now I know I can do it with Windows.  I'm going to pick up some more of these 64GB cards and make some more raw system cards for my Jetson Nano.   Next time I'll give the Linux dd command a shot and see how that works.

@robo-pi

Cool. Its good to know that windows can do that. I havent done anything with any of my nano cards so far worth saving, so i havent bothered to even try yet

In fact I'm thinking of nuking the jetbot card I'm using cuz it's working now and I'd like to start fresh without all the mistakes. Right now ive got 2 notebook folders and only one of them has the "follow me^ subroutine that is the main reason i built this thing in the first place

On the other hand, since it's working maybe i should just leave it alone

I did just nuke the ubiquityrobot on my pi3 tho, cuz i know i messed that one up, and i want to start fresh cuz I'm taking the udemy ros course, which is helping, im starting to understand how these things are built,  but im not at the point yet where i can define hardware, and i think that's the missing magic piece to this puzzle

I can control the bot,but only if i use the hardware that they proscribe. In fact, i get an error if i dont apply power to the motor board which means theres some kind of communication response from the motor board which means i cant just swap it out for the one i want

If this works tho, it'll be worth the effort

Posted by: @spyder

I havent done anything with any of my nano cards so far worth saving,

I hope to have four system backups when all is said and done.   This is why I've chosen to go with the 64GB cards instead of the 128GB cards.

Why four backups?

1. A simple pure unadulterated, but completely set up system.
2. A system for my A.I. work with Paul McWhorter's Course.
3. A system for my Kdenlive Video Editing System
4. A system for Fusion 360, KiCAD and related projects.

Of course I'll be doing other things using these same system cards too.   The first back-up is just so I don't have to start from scratch when setting up the other three systems.  The first one is to get Linux set up and configured the way I like it, no passwords being asked for, single clicking on files, the WiFi connect, etc.  Just the basic OS stuff.  It makes it a lot easier to set up the other 3 system cards with all that already out of the way.

The A.I. system will have Python, Numpy, Matplotlip, Jupyter Notebook, etc., all installed for doing A.I. work.  I'll also be installing OpenCV, and probably TensorFlow, and who knows what else?  So once I get all that stuff installed, I'll want to save that system image so I don't need to reinstall all that junk ever again. ? 

As far as any actual data files, like Python Code, etc. I can just save those to my cloud as data files.  They won't need to be backed up as an OS system image.

Then I'd like to do the same thing for Kdenlive, which will also have a lot of other related programs too, like Audacity, etc.   So once again, once I have all those programs installed and customized for my purposes, I'll back up those  system  images too so I'll never need to reinstall and reconfigure those program.  Again, actual data files like videos etc, can be backed up like regular data files.

And then finally I'll do the same thing with a Fusion 360 system which will most likely also contain other  CAD programs like KiCAD, etc.   So again, once I get that system all squared away I'll save an image of that one.  Any actual drawings I make can be saved like regular data.

So in the end I'll have four SD Cards.   One for each basic system listed above.

Otherwise, if I try to put this all on one SD card, two problems emerge. 

1. First, they start to fill up the entire SD card and clog everything up.
2. Second. if something goes wrong and I lose the system then everything goes down the tubes at once.

So I like having different system cards for these different purposes.  It keeps everything nice and neat and makes it all run faster too.

Of course, if I ever use a Jetson Nano in an actual robot, then I'll have a system image for that set up as well.  But right now, learning A.I., Making Videos, and using Fusion 360 and KiCAD are my priorities.   And I don't want them all jumbled up on one SD card.

So that's my approach.

@robo-pi

None of the cloning hardware seems to like the Nano image for some reason

It LOOKS like it's recognizing both drives, but, when I press the "clone" button, nothing happens, so I watched it for an hour, and nothing continued to happen

I then tried sticking the cards into the Pi4, and using the much loved DD procedure (with no parameters, due to the multitude of various partitions) and, it ran for a few hours, an showed me a running percentage, then actually "finished" after about 10 Gigs, but the card refuses to boot. It did look like it was progressing during bootup, but, it kept running into errors that looked like partition errors. So, even tho it was taking its sweet time, since it did look like it was making something resembling progress, and I gave it overnight to fight its digital battle, I ultimately gave up on that procedure

Of course, the first thing I tried was win32diskimager, but the problem was that windows absolutely refuses to recognize the Nano card as an actual disk the way it recognizes the Pi disks, so I can't use any software that requires windows

I also went into the workshop to try it on the computer down there, but had the same results of windows finding my Nano disk as a non-entity

My guess, and this is only a guess, is that win7 doesn't have the capabilities when it comes to reading the disk the way @robo-pi 's win10 computer does

Then I tried, just for kicks, plugging the cloning box thingamajig into the computer via the USB port, but, had the same non-recognizable results as every other time I plugged in a Nano disk

Now, the very cool thing about the Pi4 is that it has built in SD Card Copier Software built right in, which, surprisingly (and this REALLY amazed me) it was able to make a copy of ITSELF !

I didn't think that was possible due to files being locked while running, but, wow, it worked !

My only remaining idea for the Nano is to try using the Nano to use the DD command in the hopes that it would have a better chance of recognizing the garble of random partition types, which, unfortunately, I can't try at the moment due to a lack of spare 64G SD Cards

Posted by: @spyder

Of course, the first thing I tried was win32diskimager, but the problem was that windows absolutely refuses to recognize the Nano card as an actual disk the way it recognizes the Pi disks, so I can't use any software that requires windows

I don't know about Windows 7, but on my Windows 10 computer a Jetson Nano system card shows up as a whole bunch of drives.   If my current highest drive letter is say "G" when I plug in the Nano card I'll instantly get drives H,I,J,K,L,M,N,O,P and Q,

At first I had no clue what to do.  It looked like it would be impossible to clone.  But then I discovered that you only need to clone the first drive letter, in the above example it would be drive H.  And that ended up making a system image of the whole card that would then boot up in the Jetson Nano.  Of course, it's a two step process.  First the image needs to be saved to another storage device as a *.img file.  And then that image file needs to be written back out to a new SD card.  But after all that, the new SD card would boot up in the Jetson Nano.

Posted by: @spyder

Then I tried, just for kicks, plugging the cloning box thingamajig into the computer via the USB port, but, had the same non-recognizable results as every other time I plugged in a Nano disk

I'm not sure what you are doing there.  My cloning box serves two purposes.  It can either be a 2-drive bay for external hard drives for the computer.  Or it can serve as a cloning machine.   When operated as a cloning machine it doesn't need to be connected to the computers at all.  

There's a switch on it marked  PC <-> CLONE.

If the switch is in the CLONE position there's no need to have the box connected to the computer.  Just need to have two drives in it.  One in the SOURCE slot (the one you want to copy) and one in the TARGET slot (the one you want to clone to).

Then just press the START button and it clones the SOURCE onto the TARGET.

The only caveat is that the TARGET needs to be the same size or larger than the SOURCE.

This can actually be a problem for some cards that are marked the same size.  Not all SD cards have exactly the same capacity even though they are marked as being the same size.   Just a few bytes difference can prevent the cloning process.   As long as the TARGET device is the larger of the two it should clone.

In fact, I just bought a pack of (10) 16GB SD cards.   I was going to clone my current 16 GB system cards over to them.  But I can't because those new 16GB cards are a few bytes smaller! 😊 

So for these cards I'll need to manually install a system onto them, and then I can clone those systems to larger cards.   In fact, this is why it's best to set up an initial bare-bones system SD card on the smallest capacity card you can find.  They you can be sure that you can always clone it to cards that are slightly larger.

It's a shame that the manufactures didn't all get on the same page and make sure that their cards all have precisely the same number of bytes available.  But they didn't.  So cards from different manufactures have slightly more or less storage capacity than they are advertised to have.   Just a few bytes difference is all it takes to prevent one card from cloning to another.  You can always clone from a smaller card to a larger card, but never the other way around.  Even if they are only a few bytes different in actual size.

A cloning machine copies every single byte over.  So if the TARGET card is just a couple bytes too small the cloning machine reports that there is not enough space. It won't toss out even a couple bytes.  It just copies the whole thing byte-for-byte.   It doesn't try to figure out whether it can toss out any unnecessary bytes.

@robo-pi, @spyder, and @pugwash

I had already discussed my issues in trying to clone my Jetson Nano SD cards in another thread with RoboPi, but maybe here we can put some more eyes on the issue I'm encountering.

Unlike what RoboPi sees, multiple drive devices, when he attaches his Nano SD cards to his Win10 machine, I see nothing on my Win10 machines.  Two machines anyway.  I have a laptop and a desktop and neither one even sees anything. Not even a beep out of Windows that it's seen something or any messages like you need to format this card.  Now with the SD cards used in the RasPi's they always show the "boot" partition as a drive.  Which is fine as that is all Win32DiskImager needs.  I think the RasPI "boot" partition is "FAT" while the Nano uses EFS (is the right term), but why does RoboPi's Win10 see all the EFS partitions and mine see none of them?

Posted by: @robo-pi

My cloning box serves two purposes.  It can either be a 2-drive bay for external hard drives for the computer.  Or it can serve as a cloning machine.   When operated as a cloning machine it doesn't need to be connected to the computers at all.  

So if you are using your "box" why does the Win10 machine even need to enter the picture.  Maybe I need to build one of these myself.  How about reminding me what thread you discussed this "box."

Posted by: @codecage

So if you are using your "box" why does the Win10 machine even need to enter the picture.

If you are using the cloning box to clone an SD card you don't need to have any computer at all.

I bought mine from Amazon, but apparently it's no longer available.

Aukey Cloning and Docking Station

Works like charm as either a cloning machine, or a 2-drive docking station.

I pay $36.99 for it back in 2015.   Looks like similar units today are closer to $60

Posted by: @codecage

but why does RoboPi's Win10 see all the EFS partitions and mine see none of them?

I don't understand why everything seems to work differently for different people.  This kind of inconsistency is deeply disturbing.

In any case, if you recall, Paul McWhorter was complaining about his Windows machine popping up a gazillion windows every time he plugged in his Jetson Nano SD card.   Mine used to do that same thing, but I solved that problem by changing a Windows feature in the Control panel.

Now if I plug in a Nano System SD card I just get a big long list of drives.  It treats each partition as a separate drive apparently.   But I found that if you just ignore all but the first one you can still make a system image using WinDisk32.  You just ignore all the other partitions.  Apparently they are included automatically anyway.

I don't know how it all works.  I just know that so far it's been working for me.  

So knock on wood for that. 😊 

@robo-pi

I have something like what I'm seeing in the picture, but it doesn't do any "cloning" on it's own.  It only allows 3.5" and 2.5" physical drives to be connected via USB.  I'm confused, but what's new? 🤣 

@robo-pi

Well my machine never did what Paul and your machine was doing and maybe that is because whatever is keeping me from seeing anything is what stopped that reaction.  What was it you changed on your machine that stopped you from seeing that first reaction.  Maybe there is something there that might be affecting the reaction I'm see at this time.