Topic

Messages

Backing Up

We are a 10-person agency who specializes in print. As such, we generate a huge number of large computer graphics files and are struggling with our extremely complicated back-up system. We currently have several different hard drives on our server which house different clients' material and they are backed up on numerous DATs on a rotating basis. The DATs are never overwritten, but will copy a job if there is any new activity on it.

When jobs are finished, they are supposed to be archived onto CDs, but we also have a copy of the file on one of the DATs and it is also kept on the server's hard drive, which, needless to say, is always running out of space! This seems like overkill to me, but the artists are afraid to throw anything away.

How do people out there handle their backups? Also, who is in charge of actually performing the backups? What software do you use (we have Retrospect)? And finally, do you have a system for storing tapes/CDs off-site in case of a fire?

Any help would be much appreciated!

Catherine Colangelo Business Manager The Phillips Agency

Our backups are handled by our IT Department. We use Retrospect on our AppleServers and Veritas Backup Exec on our NT servers. We used to take each days tapes offsite but have recently invested in a fireproof media safe. If you go that route, make sure it is rated for media.

On archiving of completed jobs, one specific production artist is in charge of burning it to CD and then removing it from the server. The IT department will remind him as the server starts filling up. So we do not keep any jobs on the servers unless they are a work in process.

You might indeed have a case of overkill with three copies of the job, however, one lost job and having to do it all over again will pay for an awful lot of CD's, tapes and HD's.

Jim Kennedy Manager Information Systems Bill Brokaw Advertising Inc.

We are a 22 person ad/pr agency. We use Retrospect also and DLT tapes to back up all files overnight, Monday thru Friday. We reuse the same five tapes week after week and keep a couple of spare tapes on hand. I am the director of operations and do this operation each day and take the two most recent backup tapes home with me in the evenings.

We also archive our creative files on CD-ROMs. The traffic manager is responsible for this operation as she is the best person to know when jobs are completed and ready to archive. She keeps a log of which files have been archived and when. Another staff person keeps these discs off site for us. After the file is archived on CD-ROM, it is deleted from the file server.

All staff members are instructed to save files to the server so a backup copy is made every evening. However, if they choose to save files to their hard drive, they are responsible for backing up those files.

There have been several instances during the past two years when I have had to retrieve files from the tape backups, usually only a day or two back. There has only been one instance when we had to retrieve a file from a CD-ROM.

Marilyn Power Director of Operations Stuart Bacon

We have an almost identical setup - 25 employees specializing in print. All files are kept on a RAID attached to an ASIP fileserver. There are nightly incremental tape backups M-Th. Friday tapes are rotated so each of the 4 only gets used once per 4 weeks. Our backup software is Retrospect. The previous night's backup is taken off-site at the end of each day. Other tapes are kept in a firesafe.

Completed jobs are burnt to CD. One copy is kept on-site for archival reference. The other is taken off-site. All tape switching, CD-burning etc. is performed by the systems administrator (me), but documentation is posted so that anyone could do it. One person is officially designated as backup if I'm not around.

This system is not ideal, but it has a lot of redundancy and it keeps everyone happy.

Drin Gyuk Systems Administrator Piper Studios, Inc.

Storage systems are like houses, no matter how big they are you always manage to fill them up. Our 23 person shop also does a lot of print work and hence deals with the resultant huge file issues. Some things have helped: We have a 175 gb jukebox type server that holds all work. This is backed up to DAT automatically every night. We also use Retrospect. The backups are stored in a fireproof, locked cabinet. Copies are also stored off-site on a weekly basis.

Completed work is archived at the end of every month. We use DVD-ROM for this. Capacity is 5.2 gb per disc. It's fast, easy to access if you have to go back and get something, and cheap! We use conventional CD-ROM to burn files for the client.

Feel free to call/e-mail if you have any other questions.

Hope this helps,

Jerry Bray, COO Harvey & Daughters

Backups should be maintained either by the IT Director, CTO or department head.

Retrospect is not really good enterprise software. We use it here (because we are a mac shop), but I curse it all the time. For more industrial (without getting crazy) Backup Exec is good (aprox $1000 for enterprise with another $500 or disaster recovery).

If people want access to file via a server, it's fairly easy to make a terrabyte server now with IDE drives (aprox $8000), that should keep em happy for a while. For something smaller setup an NT box with ide drives with Promise controllers and you could have 4-8 drives, then either RAID them or use Win2Ks dynamic storage to make one huge drive.

For tapes, you can use AIT and a changer, which will get you 50GB uncompressed, and on an automatic two week rotation. The whole rotation is removed and stored off site...so you need aprox 20 tapes. This kind of solution runs about $14000 for hardware, software, tapes.

Win2k has some built in archiving solutions (like live, backup, storage....then way off line).

Merlin Johnson Information Technology Director Lambesis, Inc.

It came to my attention only after upgrading to Clients and Profits Classic 4.1 that the "auto quit" function in Classic had been changed. Now (as confirmed by the staff at C&P), instead of actually quitting Clients and Profits at a preset time (in order to allow for a nightly backup), C&P 4.1 just warns the user with a dialog box. This means, of course, that if nobody is there to respond to the dialog box, C&P stays active and the database can't be backed up.

We've been working with our users to be more consistent about exiting C&P at the end of the day, but this is far from a perfect solution.

Has anyone else experienced problems with this change? If so, have you come up with a better solution? We're trying to develop a work-around, but I would be curious to hear feedback from other firms. Thanks.

Daniel Thompson Director of Operations Phinney/Bischoff Design House

We did not change how the program works. This feature should quit the user. It's been duplicated and reported as a problem in the program and hopefully will be fixed in the next update.

Kent Richardson Clients & Profits Support

Why can't there be some time based or admin based way to FORCE a quit?? If a transaction is started by a client what happens? Does C&P hold a record/field open the entire time an entry is made? Why can't the client hold the data in memory and only writes to the main DB when they have completed an entry? Does the OMNIS db engine support this? I know it's pretty hard to do without a real client/server based program....but, if a shop doesnt want the time and expense to operate an Oracle based system, why not build a small server program to monitor the C&P database?

It really, really blows to have your backups not work because there is no easy to way close users out of the DB. Part of this response is venting steam, but when we start to lose entire days because the DB got corrupted, then there's problems. I've thrown more hardware at the problem, but that just decreases the amount of time wasted on repairs, doesnt fix the real problem.

BTW - doing a DB repair on a Mac sucks. ESPECIALLY a 500MHz Powerbook (256MB), you'd think that would be pretty fast as I dont know if C&P supports the G4. It took over 4 HOURS (and in some repairs takes 6-8 hours!) to locally repair a 355MB DB (basically the HD in the powerbook stinks). I started experimenting at this point. I put the DB on a 7200RPM IBM IDE drive in a Firewire box....the time to repair went to 30 mins! I built a new dedicated server with some parts laying around - dual 550 P3s, UW2 SCSI, IBM 7200RPM (2MB cache) drives, 512MB, and Win2K Server. I repaired the DB remotely - ran C&P on my powerbook with DB on server...time to repair went to about 10 mins (basically using only 1 CPU as C&P is not aware of MPs and Win2k is not that good at distributing but MUCH better than NT4sp6a).....I installed C&P locally on the Win2K server...time to repair went to 4 mins! If you're gonna repair a large DB do it on the fastest HD you have.

I think at this point there needs to be some response from C&P about database design. If C&P gets into bigger shops and they run into problems with backups, they're gonna dump it. What do large transactional databases do?? (backup off the wire?) Is part of the reason for moving to Oracle because of this and the general fragility of the OMNIS engine? I've used Foxpro (ver 2.1 (DOS) up to Visual Foxpro for years and I think they are using the Jet engine, I never had as many problems with FP db corruption (I've built multi-user relational DBs) as I do with the C&P database.

We are a somewhat small shop (aprox 50 users with about 10-20 in at any one time) that puts its business on C&P...the program itself is wonderful, but something is not right with the database. WHY was the auto quit feature removed??

Without having to build some sort of silly applescript or hypercard program to try and quit out of C&P, what can you do? Retrospect's auto dismount feature doesnt work...and with C&P doing a force quit will screw the DB up.

Now I'm looking into Win2Ks user access features to see if I can force logoffs (regardless of state of DB) at particular times just so I can get a DB backup...faulty or not.

Merlin Johnson Information Technology Director Lambesis, Inc.

Same thing here with 4.0 Pro.

Our data is on NT, and you can force a disconnect with NT, but that leaves you wide open to corrupt data if you force someone off right in the middle of a write.

I would say that is why C&P made the change. To protect the data.

If you have a user that habitually isn't exiting let them know that if they don't shut down properly you will have no choice but to disable their C&P access, as backing up the data every day is of paramount importance. We had to do that once, about a two months ago, and that person hasn't forgotten since.

Jim Kennedy Manager Information Systems Bill Brokaw Advertising Inc.

We run about 20 Macs and several Windoze boxes. We use Retrospect to back everything up. I've created a folder on everyone's HD called Current.Jobs. People are instructed to work out of this folder. I run 5 separate daily "Storage Sets". One for Monday, Tuesday, Wednesday ect. Using 5 storage sets for each day of the week has several advantages over using just a single storage set for the entire week. 1) I have 5 copies of everything. 2) My storage set size stays smaller so I don't have to change storage sets as often.

We don't recycle the tapes here. My feeling is the $20.00 for a tape is small expense compared to the >$1000.00 or so effort and time it would cost us to recreate a job we did back in 95. (Yes our tapes go back that far. And yes I have had to pulled stuff off tapes going back that far)

FYI - A "Storage Set" is just a list of the stuff on your tape(s). You can have a lot of tapes in one storage set. Some people prefer to change storage sets based on a certain number of tapes. Others based on a certain time frame, like every four months. Others simply recycle the storage sets and start over. There is no real criteria for when to change a storage set except that the storage set can get rather large. I change them when the set gets up to about 100MB or so.

FYI - There are three types of back ups.

Full- Copies everything in the folder. New - Not sure how this differs from full. Incremental - Compares the items in the folder to the items in the storage set and ONLY copies those items not already in the storage set. This is the method we use on our daily back ups.

Labeling - The way you label your tapes and what you call your storage sets makes a big difference in keeping organized and sane. I call my storage sets... Monday Nightly BU A Tuesday Nightly BU A Wednesday Nightly BU A ect. My storage sets get rather large in terms of the # of tapes I use with some up in the 25 tape range at the moment. Therefore the 20Th tape in Monday Nightly BU A would be called.... 20 - Monday Nightly BU A

When my storage set gets up to 100MB I start a new one called... Monday Nightly BU B. Then C, D E ect...

We keep our tapes in a special kind of Fire safe rated for Media. Ordinary fire safes, I'm told, won't do. Never had the misfortune of testing this so I don't really know.

Job #s Getting stuff on tape is the easy part. Finding it however is a little more difficult. It is essential you adopt a job # policy and make sure your people stick to it. Put a job # on everything. Folders, files and even in the pictures themselves. You can tell Retrospect to show everything you backed up on Monday the 15th. But it is far easier to have Retrospect to search for Job # 4444 on Mondays Storage set.

Archiving This email is getting a little long but I have a lot to tell you about archiving as well. But to keep it brief I will just say we have a very robust Archiving and (More Importantly) Retrieval system that involves a FileServer, burning CD's, Using Cantos' Cumulus, a web server, AppleScript and Filemaker Pro. Our design staff likes the ability to search the archives in a web browser, actually see the job they want, (Instead of just a name) find what CD it is on and go grab it off the shelf themselves. I like the fact that I don't have to sit around all day waiting for someone to ask me for something.

All our CD's are backed up to Tape so if one goes south or gets coffee spilled on it I can recreate it. Cumulus is a thumbnail database identical to what you would see at Photodisk or ToniStone. Most of our jobs are individual pictures so archiving is easy. But when we have a Quark file we have to make a PDF first. Canto won't display a Quark file. All these programs are automated, in at least some aspects, with AppleScript.

Labor - I do it all. In addition to Backup guy, Archivist, I'm Network admin, system admin, tech support, desktop support an

User

Our system administrator has developed an Applescript program to go out and quit C&P before initiating a Retrospect backup. It hasn't been fully tested yet, but it looks like it will work, at least for the interim. It probably wouldn't be a good solution for a very large or cross-platform shop, though.

Daniel Thompson Director of Operations Phinney/Bischoff Design House

Well, yes... if the program was quitting people while the db was being written to, then the feature needed to be fixed. It can't be that hard -- if a *user* exits the program during a write, the program waits until the write is complete before quitting (doesn't it? -- or is that another potential source of corruption?). Clearly the correct way to implement the feature would have been to remotely tell the client to start its exit routine, rather than just instantly cutting in and shutting it down. Correctly implemented, the feature would be no more dangerous than giving users the ability to quit whenever they like.

But yes, that's the same reason we're hacking together a remote-quit AppleScript instead of just shutting down the ASIP server at a certain time. It's safer.

Daniel Thompson Director of Operations Phinney/Bischoff Design House

I have our database on a Apple Workgroup 8550 server, on it's own drive, running AppleShare IP 6.2, OS 8.6. I have 20 Macs and 3 PCs accessing the database. We use Retrospect backup to DAT tape. The only problem I have encountered is being unable to back it up with users logged on. Periodically I make a copy to ZIP and one elsewhere.

I am making sure everyone is off C&P this evening and see if it gets backed up properly.

Matthew P. Hegge Technology Officer/Graphics Specialist Studio One, Inc.

Has the Retrospect/C&P Data problem been solved yet? We just got burned by this. We thought our data was being backed up, but Retrospect was only getting it when ALL users had logged out (rarely). Any C&P or AppleScript solutions available?

Grant Davis

Issues with Auto Quit

After enabling the auto quit feature we get the following message on our Macs at C&P startup.

"Clients & Profits doesn't have the proper files installed for the automatic quit feature to function properly. check the C&P website for more information."

Joel Koenigs

We use the auto quit feature and are running Pro 4.07. When one of our PCs running Windows 2000 NT reloaded C+P 4.07 he is now getting an error message each time he launches C+P. It reads, "Clients & Profits doesn't have the proper files installed for the automatic quit feature to function properly. check the C&P website for more information." I checked the Web site and there is no information to help me out! Does anyone know the solution?

Julie Townsend