Posted by Michael Krantz, Google Blog
Team
– via craigslist, and thanks for asking. Our engineers, though,
tend to come by more varied, and occasionally odder, routes. Some
get recruited out of grad school, or by friends or former
colleagues. Others just send their resumes to jobs@google.com. For
a few engineers, though, the path has been more interesting.
href="http://www.flatfeetpete.com/musicbox/index.html" >Peter
Bradshaw, for instance, built “a music playing system based on
printed cards with barcodes and webcams. Includes lego!” (No, I
don’t know what that means, either.) Over the next few weeks, we’re
going to post some of their stories.
Like this one, from Systems Administrator Aaron Joyner:
onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}"
href="http://googleblog.blogspot.com/uploaded_images/AaronJoyner-790730.jpg" >
alt="" border="0" />
My story started when I came into work one morning and was unable
to look up something on Google. Being the sysadmin for my company
at the time, it was my responsibility to resolve the problem, so I
started poking around. It turned out that our DNS server [ed: all
the jargony stuff you'll hear in this anecdote refers to the
software that websites use to connect and talk to each other] was
returning an error when trying to look up google.com, specifically
a server failure error. Just as I’d convinced myself that it
wasn't our problem but Google’s, the problem suddenly resolved
itself. I promptly forgot about it and went back to my regular
work.
But then I came in the next morning and had exactly the same
problem, so I started looking at Google's DNS responses very
closely. It became clear that the specific combination of
delegations and glue records they were returning [ed: see note
above] would result in an eventual error approximately once per
day, and this would then take it about five minutes to give up and
try again. Not entirely convinced that I should point the finger at
Google yet, I posted
href="http://www.trilug.org/pipermail/trilug/Week-of-Mon-20050328/033828.html" >
a message to my local Linux Users Group asking if anyone had
had problems with resolving google.com addresses and got a couple
"Yeah, I did have a problem like that once recently"
responses.
Thus reinforced, I headed over to Google.com, found the
"Contact Us" page and the "Report a problem"
link, chunked in a brief problem description and a link to the
href="http://www.trilug.org/pipermail/trilug/Week-of-Mon-20050328/033838.html" >
archived copy of the long technical description from that same
mailing list thread, and thought to myself, "Gee, I'll
never hear about that again." But then one afternoon a week
later I get an email that said, basically, "We've received
your problem report, and forwarded it on to the appropriate
department, if they need any further information they’ll contact
you. Thanks." Again, I thought, "Gee, how nice. I'll
never hear about that again."
But that evening I got an email from Dave Presotto (the guy who
wrote the DNS server for Plan9) saying that he was looking into it
and would get back to me. Forty-five minutes later I got another
email, this one describing how he believed they had accidentally
fixed the problem earlier in the week due to general code cleanup,
and asking what I thought of the solution. After I recovered my
senses and stopped bouncing around the room, I had a few email
exchanges with Dave, in the course of which I asked casually if
they needed any good sysadmins out in Mountain View. He referred
me, and the rest is history.
Tags: , Blog, google, Krantz, Michael, Posted, Team, via
alt="" border="0" />
alt="" border="0" />