The content of the article
Recovering data from non-working flash drives is a frequent request with which they turn to the "tyzhprogrammer". Let's see what can be done with a "killed" flash drive using utilities such as TestDisk and PhotoRec, and then without them – using a Hex editor and a head on your shoulders.
Recently a friend came to me with the phrase: “My flash drive is broken, can you see it? In principle, if it doesn’t work, then okay, but there are several files, copies of which are not present. ”
Of course, I took the flash drive and promised to see what could be done. It's a sin not to help a friend! The input data was as follows: "Windows stopped seeing the USB flash drive." I didn’t get any other intelligible explanation of what happened.
And now, when I had some free time, it was time to try to recover some data from the flash drive.
This article discusses flash drive recovery in Linux environment. In Windows, you can also recover data: there are various utilities and proprietary products (for example, R-Studio), but this is a topic for separate articles.
First of all, connecting the USB flash drive to a laptop with Linux, I made sure that the hardware of the device is alive, and it is the data on it that is damaged.
The second thing I did was shoot the image.
Safety precautions: shooting the image
The most important part in data recovery is not to ditch even more data with your actions. All the actions described in the article were performed exclusively with the image of a flash drive. You can take the image with the following commands (of course, you need to specify the path to your device):
$ dd if=/dev/sdc of=flash.img bs=512
Alternatively, you can use the ddrescue command:
$ ddrescue /dev/sdc flash.img /tmp/flash.log
Personally, I prefer the second method, since ddrescue tries to read the data in several passes, and also (if you gave the command to write a log) to interrupt the reading and continue from where it stopped. Plus, the utility gives a nice report on how much data was counted and how much was not, and an estimate of the time until the end of the image capture.
It also makes sense to work with a copy of the image. Suddenly you spoil it, and it's not a fact that you will be able to remove the image from the flash drive again if it dies due to hardware problems. For partial copies of the image and restoring damaged parts to their initial state, I recommend using the same almighty dd.
$ ddrescue flash.img backup_part.img bs=10M count=1$ ddrescue backup_part.img flash.img conv=notrunc
The notrunc parameter is needed so that dd does not truncate the destination file when the data in the source file runs out.
After removing the image of the flash drive, I looked at the contents. What I saw surprised me a little.
$ hexdump -C flash.img|less
00000000 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff | ……………. |
00400000 01 76 0a 00 02 76 0a 00 03 76 0a 00 04 76 0a 00 | .v … v … v … v .. |
00400010 05 76 0a 00 06 76 0a 00 07 76 0a 00 08 76 0a 00 | .v … v … v … v .. |
00400020 09 76 0a 00 0a 76 0a 00 0b 76 0a 00 0c 76 0a 00 | .v … v … v … v .. |
00400030 0d 76 0a 00 0e 76 0a 00 0f 76 0a 00 10 76 0a 00 | .v … v … v … v .. |
00400040 11 76 0a 00 12 76 0a 00 13 76 0a 00 14 76 0a 00 | .v … v … v … v .. |
In the image, the first 4 MB of data was filled with 0xFF. Is the flash unit damaged? Someone's trying to erase data? An application crashed? Why the area is overwritten is irrelevant. The main thing is that we have neither a partition table nor a file system structure … Although if you look closely, you can see a pattern. Before us is a sequence of 32-bit numbers increasing by one (in the format LittleEndian): 0x000a7601, 0x000a7602, 0x000a7603 … Therefore, we most likely had a file system on the flash drive FAT32…
Well, let's try to recover the data. Let's start with the TestDisk utility.
TestDisk – not just a utility, but a powerful data recovery harvester.
TestDisk is developed by Christophe Grenier and is distributed under the GPL v2 license. This utility is designed primarily for recovering lost partitions on storage media, as well as for recovering the boot sector.
- fix the partition table, recover deleted partitions;
- restore FAT32 boot sector from backup;
- rebuild (reverse engineer) the FAT12 / FAT16 / FAT32 boot sector;
- fix the FAT table;
- rebuild (reverse engineer) the NTFS boot sector;
- restore NTFS boot sector from backup;
- repair MFT;
- define a backup SuperBlock ext2 / ext3 / ext4;
- recover deleted files on FAT, NTFS and ext2 file systems;
- copy files from remote FAT, NTFS and ext2 / ext3 / ext4 partitions.
Run TestDisk with the following command:
$ testdisk flash.img
We see the menu.
Select the menu items Procced → Intel → Analyze and get the following.
We see that TestDisk did not find the partition table. Expected, because it is overwritten. Let's try to restore it using the "quick search" of partitions on the disk. Select the item Quick Search.
TestDisk didn't find anything, but that's to be expected, because the FAT32 partition is also damaged. TestDisk now offers us to register the partitions manually, but we do not know what was where. So let's put this utility aside for now. To exit, just press q several times.
Well, then let's take another invention of the same author – PhotoRec.
PhotoRec Is a program for recovering lost (deleted) files. It was originally developed to recover images from the memory of digital cameras, hence the name – PHOTO RECovery. Over time, it was overgrown with recovery functions for other types of data, but the name remained.
PhotoRec looks for known file headers. If there is no fragmentation, which is often the case, it can recover the entire file. PhotoRec recognizes numerous file formats, including ZIP, Office, PDF, HTML, JPEG, and other graphic file formats. The complete list of formats supported by PhotoRec contains over 390 extensions (about 225 format families).
If the data is not fragmented, the recovered file must be the same size or larger than the original file. In some cases PhotoRec can find out the original file size from the header, so the recovered file is truncated to the required size. However, if the recovered file ends earlier than its header indicates, it is discarded. Some files, such as MP3, are data streams. In this case PhotoRec analyzes the received data and then stops recovery when the stream ends.
Let's set this utility on our flash drive image and see what happens.
$ photorec flash.img
We see the already familiar interface, select Proceed → Search → Other, specify the folder where to save (it is better to create it in advance), press the c button. And we are waiting.
As a result, we get several folders with thousands of files in them.
A quick inspection showed that some files were restored: documents, pictures, and sources. But there are no file names, no date of their creation, no folder structure. In addition, as it turned out, there was some kind of documentation on the flash drive in the form of HTML pages with a bunch of small pictures. In this connection, the search for valuable files would take more than one hour …
And, as indicated in the sidebar, the fragmented files either did not recover, or are damaged (truncated).
Apparently, you will have to strain all your strength and manually restore the FAT32 structure.
To restore the FAT32 structure, you need to carefully read the documentation, calculate the values of the key parameters, and then enter them into the FAT32 boot record. Briefly the essence of the FAT32 structure is shown in the figure.
This includes the boot sector, the FSInfo structure, two copies of the FAT tables, and the data area. The boot sector (aka BPB – Boot Parameter Block) contains basic data that describe the characteristics of the partition, and the bootloader code.
The FAT table stores the records of the numbers of the next clusters in the file / directory chain, the attribute of the last cluster in the chain (value 0xFFFFFFFF) or the attribute of a free cluster (value 0). The data area starts from the root directory, the contents of the further area depends on the data in the root directory entries and the corresponding chains of the FAT table. For a more detailed description of the file system, see the links given in the sidebar.
For convenient work with the image, we need a Hex editor. Personally, I really like the editor 010 Editor… It allows you to define structure templates in a C-like language and highlight the structure fields in the editor.
Let's open our flash drive image in it.
Looking for displacements
To begin with, we need to calculate the addresses where the FAT32 partition and the first copy of the FAT table begin.
First, let's understand if our first copy of FAT is damaged or both. We know from the documentation that the FAT table starts with the sequence F8 FF FF FF (number 0xFFFFFFF8 in Little Endian). Let's look for her.
We were lucky – such a signature was found. This means that only the first copy of the FAT table is damaged and we can copy the data from the second table to the first. Of course, it is worth remembering that if the flash drive was disconnected suddenly, then the second copy may not completely coincide with the first (the changes were simply not saved in it). But still, we will be able to recover more data than using PhotoRec alone. At a minimum, we will get additional file names, dates of their creation, correct chains for fragmented files and even a directory structure.
We look at the address – 0x8AE400. This is the address of the beginning of the second copy of the table. Now we need to calculate the length of the table itself. You can, of course, flip through the dump with your hands until we notice the data in the root directory. But there is a simpler option. Since these are two copies, the record with which the chunk of the first copy of the table begins should also be in the second copy. And the difference between them will be the size!
Let's look for a sequence 01 76 0A 00that we saw at the beginning when we used hexdump. Options quickly begin to emerge. Stop the search by pressing ESC – we are interested in the first two occurrences.
The first occurrence (address 0x400000) is the first surviving record in the first copy of the FAT. In front of her is a blurred space.
The second occurrence (at 0xB4BC00) is the same entry in the second copy of FAT. In front of it we see the preserved data of the chains.
Let's calculate the size of the FAT table: 0xB4BC00 – 0x400000 = 0x74BC00 bytes. Therefore, if we subtract this size from the address of the beginning of the second copy of the table, we get the address of the beginning of the first copy: 0x8AE400 – 0x74BC00 = 0x162800.
So, we have the offset of the beginning of the FAT tables. Now we need to find the address of the beginning of the section. According to the data sheets and articles in the sidebar, usually the first copy of the table starts at the 32nd sector. Let me remind you that the sectors are 512 bytes each, which means that the beginning of the section should be located at the address 0x162800 – 32 * 512 = 0x15E800.
By the way, knowing the size of the tables and the offsets of their beginning, we can find the address of the beginning of the root directory.
The root directory offset is 0x15E800 + 32 * 512 + 2 * 0x74BC00 = 0xFFA000. And it begins with a Transcend entry, which is obviously a section mark.
Fine. We know the offsets of the tables, the root directory and the address of the beginning of the section, it remains to figure out what to write to the boot record. You can sit and read the specs, calculating each value. And I propose to make a knight's move! Create an empty file the size of a section. Next, we'll format it to FAT32. Then copy the first 32 sectors into our image – and you're done! 🙂
Let's try to bring this plan to life.
Continuation is available only to members
Materials from the latest issues become available separately only two months after publication. To continue reading, you must become a member of the "Xakep.ru" community.
Join the Xakep.ru community!
Membership in the community during the specified period will open you access to ALL materials of "Hacker", will allow you to download issues in PDF, turn off ads on the site and increase your personal cumulative discount!
I am already a member of "Xakep.ru"