RockyMountainPirate's posts wiped out

Posted by sdesapio 10 years ago to The Gulch: General
27 comments | Share | Best of... | Flag

I'll try and make this as clear and concise as possible...

THE SHORT VERSION
This morning, RockyMountainPirate, one of our most revered and active Gulch Producers, had her account and all of her posts deleted. The error was mine alone. There is nothing wrong with the system. We were not hacked. I deleted all of the posts accidentally.

THE LONG VERSION
I went to LA this Wednesday to watch the Director's cut of the film. After watching the film, I was in meetings the rest of Wednesday and Thursday until I went to the airport to catch a late flight back to Texas. I was effectively out of pocket for two days - no time to check emails, etc.

Waking up Friday morning I had a TON of stuff to catch up on; made my coffee, went to work; checked my email and the first thing I saw, still with one eye open, was a flagged comment. I went to check out the comment and sure enough, it was a spammer.

Here's where it gets interesting...

First, you need to know that admins of GGO have special links attached to each comment that no one else has - "Hide", "Edit", "Delete", and the new "Expunge." Expunge is a quick way to wipe out an account and all of its related posts. This is what we use for spammers.

Back to the story... right underneath the spammer's comment, was a comment from our beloved RockyMountainPirate. Both comments of course carried an "Expunge" link. Guess which one I clicked on. Right. Pirate's. The system actually asked "Do you really want to expunge RockyMountainPirate." With one eye open, I clicked "yes"... and immediately proceeded to call Jay.

Me: Jay, I just expunged RockyMountainPirate.
Jay: Oh boy.
Me: Long story. Please tell me we can get everything back.
Jay: Give me a minute.

Jay took a minute and unfortunately, there was no way to recover the account from expungement. All of Rocky's posts were lost for good.

I immediately called and emailed Pirate to tell her the news, give her free lifetime membership to the Gulch and invite her to the Atlas Shrugged 3 World Premiere in Vegas (it doesn't make up for the mistake, but hopefully it takes the edge off of the sting a bit).

So, to be clear, we were not hacked. The mistake was mine. The system worked as designed. We will be adding some further safeguards to ensure nothing like this happens in the future.

My apologies to everyone, ESPECIALLY PIRATE, for the negligence and the resulting loss of great content.


Add Comment

FORMATTING HELP

All Comments Hide marked as read Mark all as read

  • Posted by $ rockymountainpirate 10 years ago
    Thank you for the explanation Scott. Apology accepted.

    Stuff happens and I thank you for quickly contacting me and letting me know. I'll get busy finding more things to stimulate discussions.
    Reply | Mark as read | Best of... | Permalink  
  • Posted by $ stargeezer 10 years ago
    For free tickets to the AS3 premier, feel free to wipe me out ANYTIME!

    On a serious note, it's nice to be onboard with first rate folks like these here. Accidents happen and not trying to speak for Pirate, but knowing that the fist move you made after discovering your error was fessing up to the mistake and moving to makeup for the error. Very well done.
    Reply | Mark as read | Best of... | Permalink  
    • Posted by Zero 10 years ago
      Hear, hear.
      I'd expect no less from an Obj, but in this day and age it's downright refreshing.

      RMP - sorry for your (our) loss. It was good stuff. More to come, I'm sure.
      Reply | Mark as read | Parent | Best of... | Permalink  
  • Posted by $ AJAshinoff 10 years ago
    Class act plainly stating and owning up for what happened. Its much appreciated. In the future please feel free to accidentally wipe all of my post if it gets me an invite the World Premiere in Vegas :) j/k
    Reply | Mark as read | Best of... | Permalink  
  • Posted by SRS66East 10 years ago
    I respect the way this was handled. I am glad to see RMpirate is okay with it as well. So glad to see this was handled openly and the person responsible got out in front of it and took responsibility. Kudos to you Scott.
    Reply | Mark as read | Best of... | Permalink  
  • Posted by Eudaimonia 10 years ago
    Hey, Scott, if you need a nightly backup system installed, you have my number.
    Reply | Mark as read | Best of... | Permalink  
    • Posted by 10 years ago
      Because Galt's Gulch Online runs on top of Google App Engine, traditional backups are, in a sense, unnecessary. The data is constantly replicated across multiple servers automatically - so it's essentially impossible for this type of data loss to occur without you going into the system and explicitly deleting it on purpose. Which is basically what happened.

      As a fail safe however, we are looking into implementing an interval based historical backup.
      Reply | Mark as read | Parent | Best of... | Permalink  
  • Posted by khalling 10 years ago
    I know you have been dealing with those spammers alot. it's a real problem. sorry this happened. but it could happen and thanks for being vigilent. but poor pirate. ;( that's one hell of a super signed poster and DVD and J.A. handshake...
    Reply | Mark as read | Best of... | Permalink  
  • Posted by iroseland 10 years ago
    Doh! Sometimes the unexpected will happen. It what someone does after the unexpected that really matters. So, well done.. as for the convince of a quick kill option. You might want to consider adding a verify, or push it onto the database by only marking the posts as inactive in the database. Thought that may involve more query surgery.
    Reply | Mark as read | Best of... | Permalink  

FORMATTING HELP

  • Comment hidden. Undo