[Alex Pepple]

Dear Eratosphereans,

For those of you who want the quick version, the server was hacked on the night of March 8, 2008. The hackers left the system in tatters, and it took me weeks of tedious manual restoration -- since the automated tools no longer worked -- to transfer everything out of the corrupted server onto brand new hardware, new operating system software, and then piece the system and associated data back together, and finally, get it all to work again as expected. The good news is that everything should function as expected, and all your posting and data should be as before the breach occurred.

For those of you interested in the details, read on ...

The criminals essentially succeeded in wanton vandalism of not only the web pages, but of the core server files and programs as well. They essentially left a hopelessly corrupted system in their trail. The hackers wiped out all the server log files in an attempt to evade detection. Still, I uncovered the hack tools they had used for their dastardly deeds and was able to analyze the contents. The result of my investigation pointed to some group in Morocco, using an Italian hotmail address, with the claim is that it's for “dying Palestinians” and for “Peace”. They managed to deface the majority of the site’s index and home pages with their so-called message, essentially rendering it unusable. It’s possible some of you might have stumbled on these messages before the site was taken offline for recovery. I can't tell if the claim of Palestinian association is genuine, or simply a case of hackers elsewhere assuming a false identity to throw off detection attempts ... but then, there is the definite Morroco tie I uncovered.

It was a daunting task trying to piece the system software, configuration, and user data together. Indeed, it was the equivalent of trying to piece a shredded document together -- the automated restoration tools no longer worked so, I had to piece everything together manually, to salvage something close to what we had before. At some point, it looked like more like I should discard the current forum and its data and simply install a new one without any data and start anew. However, continued perseverance and determination paid off in the long run. Still, the bad news in the financial column is that I had to abandon the old server due to the unworkable amount of system corruption and, the potential time bombs the hacker seemed to have left behind. Indeed, I had to order brand new server hardware, with a clean, new disk and, with the latest operating system software before I could start with the restoration work. (New operating system software is one more way to achieve enhanced security given that new releases usually feature security patches.) The data center where the server is hosted was quite slow to react. It should have taken them a couple of hours to complete the setup of the new hardware and software as I specified to them. Instead, it took them 3 days because they kept getting it wrong several times and had to redo it each time – effectively adding that many days to our total downtime!

About the new server, I finally caved in and bought a RAID disk mirroring setup, notwithstanding the extra initial and monthly costs involved. I am currently looking into subscribing to an automated offsite backup service for even more data redundancy and faster recovery from any future server compromise. Ideally, dual redundant servers would be the ideal, but what I have now is the next best thing, providing a reasonable cost and function compromise. The next phase I’m looking into for Eratosphere, once it has been sufficiently restored and stabilized, is to upgrade to a new generation of forum software which should not only provide better resistance to hacking, but also, enhanced features and performance.
For now, I believe the Sphere is mostly back to normal. All your posts should be there with no loss of data. Still, there’s always the possibility that some features remain broken and I, somehow, missed them. Thus, I’m counting on all you to post any aberrant behavior you witness (relative to the pre-restoration Sphere) here and I’ll look into fixing such problems as quickly as possible.

We’ve been through quite a bit with the Sphere, but have never endured this scale of destruction and the subsequent downtime. I’m relieved, as I’m sure all Eratosphereans are, that we’re back!

Cheers,
...Alex

[Alan Sullivan]

Alex, heartfelt thanks from me and everyone for all your work to salvage the site and the community that had grown around it for all these years.

Your account is so scary that I just logged into my own site's server and started a full backup, which I haven't done in a long while.

Alan

[Tim Murphy]

God bless you Alex for your diligent efforts. I am sure the last twelve days have been hell, and from a Catholic standpoint, they should give you a free pass from Purgatory.

[Alex Pepple]

Thanks Alan & Tim! Yes, it's good that Hell is still waiting . And, Alan, back up right away - there are quite a few out there who seem to only know how to use their time and skill for destruction!

Cheers,
...Alex

[R. S. Gwynn]

Great work, Alex. I'm glad everything is back and running!

Sam

[amacrae]

Yes, heartfelt thanks, Alex.

Austin

[David Rosenthal]

Alex,

Thank you for all your work in getting things back on line. Eratosphere means a lot to a lot of people. I suppose I should have expressed gratitude for its existence long ago, before this event, but I hope late is better than never. I am grateful to you, the Able Muse staff, and all the staff and moderators on the Sphere for making this place available to the likes of me.

Best,

David R.

[Henrietta kelly]

I can imagine how you felt
Frustrated that your site was ruined but elated that you beat the bastids in ability to mend what they thought was too easy to break. You found the country you found the sever, Interpol needs to be informed about the vandalism who knows is this was a test run before a world banking system takes a hit by a bunch of twits

Is there any way I can help with the financial costs? I don't mind coughing up a few bucks -- as long as I get a guarantee in bright red blood that M C is going to be extra nice to me at all times LOL

regards ~~Joan

[Jan Iwaszkiewicz]

Until this happened I did not realise how much that I cared for this place.

My heartfelt thanks Alex.

Jan

[Alex Pepple]

And thank everyone. Your support is greatly appreciated!

...Alex