Recovering lost files

[dingwall]

This is a general computing question for some of the more tech savy guys out there.

I'm in a real mess. Up until yesterday my hard drive was partitioned with programs on the C partition and files on the D. Yesterday Windows (XP Pro) had trouble loading, when it did the D partition was missing. A virus scan turned up one Trojan horse. After cleaning that up, shutting down and re-starting,Windows recognized the D partition in name only - it and the files it used to contain could not be accessed. Looking in My Computer Properties, the D is still missing and the C is now the size of the whole drive.

Now for the bad news. The tech who installed XP on my system turned off the restore feature so doing a system restore is out of the question.

Semi-fortunately I have a fresh backup, however the backup seemed to fail on all my AutoCAD files and most, but not all of my Office files. I kept my Vector and AutoCAD files in the same folder, the Vector files backed up the AutoCADs didn't.

Is there any hope of recovering the lost files?

[mikejohn]

Sheldon
Luckily, Windows doesn't actually erase the data until it reuses the disk space formerly occupied by the deleted file, so recovering the data is possible.

I have a son who does this sort of thing. His mother once delated her entire university thesis files by error, and he got 98% back. He had to do it manually though. Apparently, its all down to restoring the file header.
If its copied over, however, it's lost.
You are going to need to find a local wiz-kid, I guess.
........Mike

[fleinbach]

Sheldon

There is software that will recover your data. I believe it was made by Ontrack, I believe it was called Easy Recovery. I used it numerous times to recover data from formated drives and it worked great. I will look for it when I return. I need to leave right now.

[kivimagi]

Sheldon,

I used Active File Recovery when my son knocked over my tower, and somehow we were unable to read the data in the traditional manner.
(p.s. I now have a RAID setup, so everything should be backed up automatically on another drive.)

You can find it here
http://www.file-recovery.net/

download a demo to see if it works, and then purchase if you like what you see.

Regards,

Ryan

[dingwall]

Thanks for all the input guys. I've got my work cut out for me.

I'm looking at a Raid system now. I was surprised, they're actually quite reasonable in price these days.

[richards]

Raid systems can work well if they're set up correctly. However, they are often overkill for most small businesses - except for raid 1.

When planning a raid array, you need to realized that there are several levels of raid.

Raid 0 is used for fast access with low reliability. We use that when it's important to quickly transfer data to/from the disks. Since parts of the data in each file are recorded onto two or more disks - without redundancy or parity - loss of any one disk in the array means that all data is lost.

Raid 1 is usually the most secure on home/small office systems. All data is mirrored onto two drives. That means that when data is written to the drives, the same piece of data is written onto two drives. When data is read from the drives, it is retrieved from whichever of the two drives responds fastest (usually - depends somewhat on the raid hardware/software used).

Raid 5 stripes the data onto three or more disks, but the raid array is built so that if any *one* drive fails, a new drive can be inserted into the array and the data can be regenerated from the other drives in the array. Some raid 5 arrays have drive(s) that are 'hot spares' that automatically activate when another drive in the array fails.

There are some 'gotchas', even with raid 5. In some systems, if the raid controller card fails, inserting a new controller card requires that the drives be re-formatted - meaning that all data would be erased. Speed can be a problem. Some controllers/software take their time generating the necessary parity/redundancy data - especially when using raid 5.

Raid has its place. In the photo industry, where many of the images are digitized, raid has become a necessity. High speed film scanners can generate gigabytes of data every few minutes, meaning that the transfer speed requires a varient of raid 0. However, the data is transfered from the raid 0 array to a raid 5 array before it is released to the image enhancing stations where the images are corrected for color and density. Because of the amount of data stored online at one time - terrabytes - raid 5 is the preferred configuration. Smart photolab managers insist on having their raid 5 arrays mirrored - raid 50, as a more secure way to handle the data.

For normal shop/office use, I would NOT recommend a raid system because of the cost and maintenance required - except raid 1 (mirroring). Knowing that a raid configuration can lead to a false sense of security, I don't use raid in my own business. I backup my data onto two CDs, and store all important active data on two pysically separate networked computers.

-Mike

[bill.young]

Hey Mike,

Good info...thanks. Would any of the RAID systems have helped in Sheldon's case with a trojan or virus, or would it have just ended up the second hard drive and caused the same problem with it?

Bill

[richards]

Bill,

Unfortunatly, there is nothing generically built into a RAID system that prevents a virus. Depending on the virus, and on the RAID setup, all of the disks in a raid system would be infected.

I have one unbendable, unbreakable rule that I use whenever I'm on the Inernet. And that is simply this: I never connect to the Internet with any computer that is networked. In addition, any files that are downloaded from the Internet are always virus checked before they're moved to another computer. (Be aware that checking for a virus only catches SOME of the infections.)

In my own office/shop/home, I ALWAYS connect to the Internet from a Linux based computer. The Linux OS, to this point at least, has been immune to the virus nonsense directed towards Microsoft based computers.

-Mike

[bill.young]

Hey Mike,

Thanks again. Here's a somewhat off-topic question. You mentioned backing up to CD. You hear stories of writeable CDs failing after a couple of years...any thoughts on long-term storage in general? I have a Rubbermaid tub full of floppy disks with old ShopBot files on them but find that some are becoming un-readable. Hard drives are cheap, but will they last as a long-term backup medium?

The reason I'm thinking about this now is that a friend volunteers for a local Museum http://www.barrierislandscenter.com/ and they are trying to scan and archive a bunch of old photographs that have been lent to them. He asked about a safe and long-term way to store them and the only really safe system that I could think of was to print a copy on archival paper so that they could re-scan if their backup failed. That can get awfully expensive...any other options?

Sorry for the topic diversion,
Bill

[richards]

Bill,

You've heard correctly. CDs do fail after a few years. It's been a long time since I read up on the manufacture of a CD diskette, but basically the cause of CD failures is that the diskette can delaminate over time. The problem occurs over all brands of diskettes, both pricey and inexpensive. It has something to do with manufacturing conditions and something to do with normal air-borne pollutants. From what I remember, failures start appearing five or more years after the diskettes were initially written. Of course failure could happen at any point, especially if the diskette were scratched or handled improperly. Even fingerprints can cause premature failure due to the acids and oils that are part of a fingerprint.

To avoid the problem, it's good practice to copy a CD onto a new CD every two years or so. Typically during that time span, most computers are upgraded with new CD drives and software, which helps eliminate the problem of worn out hardware contributing to the problem.

As far as scanning photographs and then printing the images on archival paper goes, you'll have a complete set of 'archival quality' problems to deal with. Processing photographs to archival standards is more a myth than a reality. Even the slightest amount of chemical contamination left on the print will eventually cause problems. Here in the West, the PH of the water is often 8.0 or higher, meaning that one would have to use distilled water as the final wash. Most achival standards (yes, there are several standards, depending on whether you believe the paper manufacturer or the museum or the ink manufacturer in the case of 'archival quality' inks) require a long wash time with frequent changes of the wash water. Unless you have your own still, the cost of using distilled water would be prohibitive. Heat and light can also greatly decrease the life time of an archival print.

So, getting back to the initial question. I would recommend scanning the photographs onto CDs, verifying the images on the CDs and then copying the CDs every few years onto new media every few years. It's always best to have two copies of everything, stored at separate locations, to minimize the possibility of loss due to fire, theft, etc.

-Mike

[mikejohn]

Bill and Mike
In the Aerial Survey industry, everything is backed up onto tape. You get a lot of storage on a single tape. These are stored in a special room, which has temperature and humidity control.
However, even this has only a 20 year life expectancy.
Mike answer seems safest, every couple of years, re-backup those files you still wish to save.
What about newer solid state type memory.
PCMCIA cards, or memory sticks? Are these any better?
............Mike

[richards]

Mike J.

In a word: "No". Solid State Memory devices comes in several forms. Most are simply 'FLASH MEMORY' or its older variant 'EPROM'. In either case, the charged cells in the memory bank can change states due to leakage. EPROM, in particular, acts as if it were a large array of charged capacitors. As time goes on, the charged capacitors slowly leak their charge until they change state. The life of an EPROM is highly affected by ultra-violet light (which is used to erase the EPROM) as well as by old-age. An EPROM that is covered with an UV resistant shield still has a useful life in the range of 5-8 years before change takes place. Flash memory is physically different, but it still 'leaks' after a period of time. 'ROM' (write once memory - should it by WOM?) is usually more stable, but not practical due to its very nature (masks are made for high-volume runs of each part).

All of the devices/methods listed in this thread depend on the assumptions (1) that a bit can be set to a certain state and (2) that once the bit has been set, that it will remain in that state forever after. The 'forever after' part is the main cause of grief. 'Forever' is a long time, much longer than the 3-8 years of many storage devices. Physical things change over time. So far, all physical things, at least the physical things that I'm familiar with in solid-state electronics, slowly change from one state to another as other forces come into play. In a perfect environment, a solid-state device, CD, EPROM, FLASH, etc., might not change, but I've never had the priviledge of living in a perfect environment.

Added to the problem of the deterioration of the actual device/material is the added problem of the availability of the reading mechanism. Drives, disk, tape, or other, are not always permanently available. Mechanical settings in the reading device (gap distances, etc.) can change or may be difficult to maintain. Calibration devices (master tapes, CDs, etc.) are also affected by the same factors that cause failure of the media being read.

The computer industry reinvents itself at least as quickly as any other industry with which I'm familiar. (I still have a few hand tools passed down from my grandfather that I could use in production without modification. However, the old IMSAI computer that I built in the late 1970's, in contrast, is merely a conversation piece. It still fires up, but it basically useless because many of the peripheral devices that I once owned have either disappeared or have themselves changed to a non-working state.)

So, the question is: "How to minimize loss?" For me, it means that I have to copy and re-copy valuable material before 'any' change takes place. I've long ago given up the search for the 'perfect storage medium' and settled for the 'realistic backup strategy'.

-Mike

[richards]

Not to keep beating a dead horse, but I have an observation. One of the main reasons that I bought a Shopbot was the fact that an ordinary computer would be used to control the Shopbot's 'controller'. Other than the one controller board in the Shopbot, everything else is basically 'off the shelf', making repair/replacement realatively easy.

I still spend much of my time repairing and re-inventing process control computers for the photo industry. At one time a company named Bremson and a company named Kodak built wonderful process control computers to control expensive photo printers ($100,000+). As markets changed and demand for sophisticated controllers waned, new design stopped and replacement parts for the aging controllers began to become difficult to find. Just last week, a customer in Boise, Idaho, asked me to fix a memory storage device for him. The stroage device, once made by Dallas Semiconductor, was no longer available. There was no substitute available. In short, a $10 part is causing a $125,000 Lucht V-7 printer to set idle during the photo lab's Christmas rush season. Re-inventing the part is possible - at the cost of several thousand dollars and several weeks down time.

For me, the solution is to throw out old electronic devices as soon as practical newer devices appear on the market. The practical life of a computer (especially components with moving parts - i.e. disk drives) is really in the 12-18 month range. They can be used for a much longer period, but depending on them to function flawlessly forever begs reality.

-Mike

[gerald_d]

"All of the devices/methods listed in this thread depend on the assumptions (1) that a bit can be set to a certain state and (2) that once the bit has been set, that it will remain in that state forever after." Well, there is a slightly older device that does meet this criteria.....the abacus.

Happy New Year All!

[mikejohn]

Congratulations Gerald. The first post of 2005.
The type of incisive, informative and useful post we need on this forum.

I'm glad any New Year resolutions haven't changed you.

A very Happy New Year to everyone
.......Mike

[fleinbach]

'ROM' (write once memory - should it by WOM?)

'ROM' stands for "read only memory".

[kivimagi]

There are a lot of engineers, ex-engineers, running shopbots.

[beacon14]

"and (2) that once the bit has been set, that it will remain in that state forever after."

Gerald - just don't knock that abacus off the shelf!

[bill.young]

And just to show that this isn't just a computer-related issue...

http://www.deadmedia.org/notes/index-cat.html

[jthelen]

I use a UBS2 external hard drive. I only connect it to backup then disconnect it. They are cheap nowdays.
Jim

[richards]

Frank-
You caught me. Here I was trying to change the world to my way of thinking and I got caught red-handed. Although I know that ROM is 'read only memory', I've been fighting that acronym since I first saw it. To me, a 'read only memory' device would always have the state of 0xFF, that is, it would be the device in its pure state, or in other words, a blank memory device, since by definition, a 'read only' device can never be written to. So, in my little way, I've been fighting that battle with my little remark about "WOM" or 'write once memory' as a truer statement of what's really going on.

Jim -
A USB external disk is a good idea. I've used a few for several months with good success.

Happy New Year to all. May it be the best year yet.

-Mike

[dingwall]

I'm a little embarassed, the partitioned drive was in my old office computer. The new one had a separate physical D drive that failed. The local disaster data recovery experts couldn't mount the drive so it looks like the only option available is to ship it off to the clean-room boys to remove the platter. The cost for a small company the size of mine seems pretty enormous, but considering the work to re-build, is money well spent.

A raid 1 system sounds good, but it seems like it's only advantage is preventing loss from a faulty drive. For damage from everything else a brain-dead bullet-proof backup system is still needed. Any suggestions for such a solution?

My last backup using the Windows backup utility didn't save the folder hierarchy so I'm left with sifting through 100s of files trying to remember their original locations. Not that I'm complaining.

I thought I had a better system using Roxio CD burning software, saving the list of folders to be backed up in the favorites, calling it up every week and burning them to a CD. It worked perfectly for the accounting files and failed miserably for the Office and Drafting files.

[fleinbach]

Mike,

Sorry about that, but I know what you mean. I too have a few querks about the way some things are done and stuborn enought to think that eventualy the rest of the world will come around to my way of thinking. Which I'm sure will eventualy happen (in some other life I suppose.)

I too agree it was a poor choice of acronym. It should have been WIOTORIF "write it once then only read it forever" but that would probaly just defeat the porpose of an acronym.

[bill.young]

Hey Sheldon,

I've never seen this product in person but it looks interesting. If I understand correctly it's basically a linux box that continuously backs up your computer over your network. It also has a web access component so that you can get your data remotely over the web.

http://www.mirra.com/

Anyone had any experience with this?
Bill

[richards]

Sheldon -

I use a very simple method. In my main drafting computer (where I have AutoCAD Lite), I have a second disk drive, drive D. As soon as I have finished a drawing, I copy it onto drive D, which is physically separated from drive C. (CAUTION, do not confuse two physical drives with two logical drives. Logical drives can reside on the same physical drive. Make sure that you have more than one physical location to prevent loss due to a drive failure.) When the file has been copied to drive D, I back it up onto storage media (CD diskette, floppy diskette, USB memory, etc.). At that point I relax a little, knowing that I have the data (drawings) on more than one physical device.

As far as protecting yourself from loss due to a virus, my only suggestion is to NOT connect to the Internet with any computer that has precious data on it. As stated in another post, I only connect to the Internet fron a computer that is isolated from the network, and I prefer using a Linux computer to further safeguard the data. (For those of you who only use Microsoft based computers, I would highly recommend adding a small Linux computer for all Internet activity. Last week I built several little Linux computers for friends/customers for less than $350 each. The computers are really quite good with a 2400 AMD CPU, 256 mb DDR ram, 80 gb hd, CD/RW, built in video, network, sound, etc. Please note that I'm not saying that you should rush out and buy another computer or am I trying to put down Microsoft or even advocating Linux. I simply use Linux (in this configuration, at least) as a virus protection computer to keep my networks clean.)

If you already have a virus on your computer, I know of no method to protect anything that you create.

-Mike

[artisan]

Much like Mike, I also keep my internet access computer and my design workstations physically separated. I transfer data with one of the usb jump drives or burn CDs for really large files. I have had to recover data from a damaged hard drive in the past and used "Drive Rescue" software to recover files and move them to another hard drive.

The internet is invaluable as this forum illustrates, But, even with the very latest anti-virus software...updated automatically and almost daily....I still get the occasional virus attack. Computers are relatively cheap these days and having a separate one for internet connections is probably the safest way to go....D

[dingwall]

Bill, that Mirra system looks real slick. I'll be looking into that in more detail.

I'm also considering a pair of portable drives to keep a set of alternating daily backups.

Other than capacity is there a clear winner between portable hard drives and solid state jump drives?

[bill.young]

Hey Sheldon,

It looked pretty slick to me too. The one thing I couldn't figure out was how you restored data from it...it doesn't sound like you can just copy off of it like a standard drive. Also it doesn't look like you can upgrade the hard drive, which might be a problem down the road.

There are a bunch of companies that sell external hard drives, and some like the Maxtors come with backup capabilities built in. You can also buy an external hard drive case from CompUSA for less than $20 with USB and Firewire, and put whatever hard drive in it you want...hard drives are awfully cheap these days. Then when it fills up just pop in a new drive and archive the old one.

Bill

[srwtlc]

The external USB/Firewire drive along with Norton Ghost v9 or Acronis True Image v8 with complete system imaging, incremental backups, and folder/file restore capabilites would be a good option also.

Scott

[richards]

Bill & Sheldon -

Regarding the Mirra system: If all you really want is networked data storage, the Mirra computer might be overkill. I just read its installation manual and it seemed like it required more from you than most file servers - and it uses .NET, which, depending on your likes/dislikes may be at the bottom of your preference list for virus-free software since it was developed by Microsoft in large part to compete with SUN's JAVA software. I'm not necessarily knocking .NET or Microsoft, and I'm certainly not knocking SUN or JAVA, but I prefer to stay as far away from Microsoft's products as I can. (My customers use a lot of Microsoft software - and hire me to fix things regularly. So, why complain when Microsoft's errors make my computer skills necessary?)

There are alternatives in the same price range that have the same capabilities - without the need to connect to an external web site, which, to me, means one less access point for external users to compromise your network. The object was to have secure backups without viruses, right?

My preferred file server system is Linux running Samba. In fact, I'm sitting a one of my customer's consoles right now loading Fedora Core 3 software on seven new computers. Each computer will be used as an additional Samba Server for their in-house network. A basic Linux computer can cost less than $300 (plus a KVM switch unless you want it to have its own Keyboard, Mouse and Video display). A Linux computer would connect to a network via an TCP/IP connection, just like the Mirra. After Samba is configured (a very simple step with Fedora Core 3), the Linux computer's shared disk(s) will appear on your existing computers just like any other shared drive. The beauty of the Linux OS is that there is NO cost for software if you download the OS from one of the hundreds of Internet sites. Of course, you can buy a boxed set of CDs from CompUSA, if you prefer. Depending on where you live, you probably have a high school student or college student living nearby who would help you set up a Linux file server either for free or for a nominal fee. Most Linux users are much like Shopbot users - neighborly and willing to help.

Now, back to the Mirra. In many ways, it looks like a Microsoft clone of the Cobalt Servers that were very popular until Cobalt was purchased by SUN Microsystems. I'm using two Cobalt RAQ2 computers as mail servers right now. The concept was new six or seven years ago, but now most people opt for a general purpose computer when they need a web server, mail server, telnet server, FTP server, SMB file server, etc. The drawback to the Cobalt, and other similar computers, was that the hardware was proprietary - meaning expensive to upgrade and repair, while a general purpose computer usually uses off-the-shelf parts from the local computer store. (Your concerns about replacing a failed disk drive in a proprietary computer are valid.)

-Mike

[dingwall]

This looks like a nice compact system. A couple of Flash drives with this backup software:

http://www.jumpvault.com/