Website outage and DNS explained • 30 March 2012 • The SnowBlog
Website outage and DNS explained
The Snowbooks website was unavailable for a few hours early this morning and it seems that it was my fault. Many apologies to our authors and those who tried to visit our site. I screwed up some 'DNS' settings and mayhem/confusion ensued.
As well as an apology I thought I'd use this as a 'learning opportunity' and explain what DNS is and why you need to know about it if you want to administer a website. Then, if you're interested, you can read about what it was I actually did wrong. So, first of all you need to know how the data whizzing backwards and forwards to a website like the one at snowbooks.com finds its way there. Much of the internet uses something called IP, which stands for Internet Protocol. It's basically a set of rules for packaging and routing data across a network. It involves taking the data you want to send, cutting it into roughly uniform chunks and putting them in the data equivalent of an envelope - or in internet parlance a 'packet'. Like any good envelope or packet it includes an address for where its contents are going and a return address saying where it's come from.
If you're trying to picture how digital data passing along a cable can be wrapped up into a packet I should explain that it's all just ones and zeroes, but the IP standard specifies a layout for those ones and zeroes. The first chunk of ones and zeroes will be the 'header' which is like the envelope. The header will contain the destination address and the return address and a few other pieces of information. Next will come the contents or 'payload' of the 'packet' i.e. the data you're trying to send across the internet, which could be something like a request to fetch the home page for the website at snowbooks.com.
But when it comes to the destination and return addresses, IP doesn't use a name like 'snowbooks.com'; it uses a number. This is similar to the way an old-style telephone doesn't let you type in the name of someone you want to talk to; it requires their phone number. So just like with telephones, the web requires a directory so that you can look up a name and get back a number that you can use to make a call. The way the directory is provided on the web is with something called 'DNS' and it's handled by your computer automatically so that you can surf the web without needing to know DNS even exists. But if it's your responsibility to add a new website to the web, you probably need to know the basics of what's going on.
The names on the web are 'domain names' (like 'snowbooks.com') and the numbers used to route data are called 'IP addresses' and most of them are four numbers, normally written separated by dots (e.g. '74.86.153.134'). You can buy the right to use a domain name by 'registering' it at one of the many Domain Name Registrars, like, say, nominet. But until that domain name has an IP address associated with it, nobody can use it to visit a website or send you e-mail.
So once you've registered yourself a domain name you need to create some DNS records (DNS actually stands for 'Domain Name System'). By 'record' what I mean is a single entry in the database of the world's DNS information. The simplest possible setup is just to create a single record that associates your domain name with the IP address of the computer running your website. That kind of record is called an 'A' record (A = Address) and you might write it out like this:
Domain name ; Type IP Address
snowbooks.com IN A 74.86.153.134
You would normally set that DNS 'A record' up with the company you registered your domain name with. They'll typically provide a control panel to use for that purpose. Where do you get the IP address from? Well, that's the IP address of the computer running your website. Assuming you're not a total DIY fanatic, you'll probably use a hosting company and rent a webserver from them. They'll tell you what the IP address of that server is and you'll create or update your 'A record' to 'point' your domain name to the webserver's IP address. That's how a domain name ('snowbooks.com') is associated with the IP address ('74.86.153.134') of a particular computer (= your webserver) on the internet.
There are other types of DNS record that serve other functions but there are only two that I want to mention. The first is the 'MX record'. That's a DNS record for routing e-mail. The idea is that you have to specify the domain name address where you want your e-mail to go because you might want it to be processed on a different computer to the one running your webserver. In our case we used to use our webserver as the destination for our e-mail, but we decided to change to using an e-mail service provided by Google - and the way in which I screwed up that change is the reason you're reading this blog post.
You can also have more than one 'MX record' in case you want to set a backup computer to handle your e-mail. 'MX records' include a number you can use to decide which computer is at the top of the list for handling e-mail, which comes next and - if you're so minded - which comes after that. Low numbers come first, higher numbers later in terms of their priority. You can create as many different MX records as you like.
The final thing to say on the matter is to explain where DNS information is stored around the web. In theory, we'd put all the DNS entries for every webserver in the world on one really big computer and everyone would connect to it to get their DNS info. That's how it used to work back in the Eighties when the internet was a few hundred computers. With billions of computers on the net, that approach won't work. Plus there'd be the problem of who was allowed to change which pieces of information. I need to be allowed to alter the DNS records for my domains but nobody else's.
The answer is twofold. Firstly, there are master computers for the web, called the Root Name Servers, but they do a fairly simple job of breaking the web into top-level domains like '.com' and '.co.uk'. They tell any querying computer where to get more information about the top-level domain they're interested in. Thus when running a DNS query to find out the IP address of snowbooks.com, you could ask the Root Name Servers where to find nameservers for the '.com' part of the name. Then you'd send a DNS query to one of those servers in order to get information specifically about snowbooks.com. Those nameservers would then respond by providing the address of what's called the 'authoritative nameserver' for that domain. That's the nameserver that is in charge of that domain and whose information can be considered to be definitive. That will typically be the nameserver of the Domain Name Registrar where you registered your domain name. In our case we registered snowbooks.com with LCN, so their computers provide the 'authoritative nameserver' for snowbooks.com: the definitive source of DNS information for our website.
You specify the 'authoritative nameserver' using another type of DNS record called an 'NS record'. That 'NS record' is what the top-level nameservers use to find the DNS information that you set in the control panel: your 'A record' and your 'MX records'.
That's part one of how the system works. Part two is the idea of 'caching'. Instead of letting you directly query one of the top-level nameservers, your ISP (Internet Service Provider) will give you the address of their own DNS servers to use for your queries. (Your router probably picks these up automatically when you connect to the internet by using a protocol called DHCP.) If your ISP's servers get asked a DNS question they don't know the answer to, they go out and ask that same question of the top-level nameservers. But once they've done that they save the answer and next time they get asked the question, they simply give the same answer as last time. Periodically those nameservers will run the query again, just to check that the answer hasn't changed because if they cached DNS results forever then changing them on the top-level nameserver would have no effect and you'd never be able to update the settings for your website. It's a combination of top-level nameservers with authoritivative nameservers, plus the idea of DNS caching, that makes the whole system work even though billions of computers are constantly seeking DNS information.
And that brings us to where I screwed up Snowbooks' setup. I wanted to stop routing our e-mail to our webserver and start routing it to Google so that we could use the Google Apps for Business service for our e-mail. What I did was to go to our Domain Name Registrar, LCN, and set up some MX records to direct our e-mail to Google's servers. What I hadn't realised was that our Domain Name Registrar wasn't the 'authoritative nameserver' for snowbooks.com. We had previously decided that our hosting company's nameservers would be authoritative for our domain. So the changes I made to the DNS records at LCN's nameserver had no effect because that's not where the DNS servers of the world were getting their information.
I waited the required 48 hours for the world's DNS servers to update their caches and (of course) found that our e-mail was still going to its old destination. So I complained to LCN. They looked into the matter and must have spotted that we were using someone else's nameserver not theirs. So in order to make my changes work they made their nameservers authoritative for our domain. But of course I hadn't created an 'A record' at LCN's servers - I assumed all that had been done long ago - I'd only created some 'MX records'. So once LCN's nameservers had become authoritative for our domain, all our e-mail used the new MX records, but there were no 'A records' to tell any web browsers where to find our website.
The situation was resolved by adding the appropriate 'A record' to LCN's nameserver and that's how you're able to read this now.
Rob
The Snowbooks website was unavailable for a few hours early this morning and it seems that it was my fault. Many apologies to our authors and those who tried to visit our site. I screwed up some 'DNS' settings and mayhem/confusion ensued.
As well as an apology I thought I'd use this as a 'learning opportunity' and explain what DNS is and why you need to know about it if you want to administer a website. Then, if you're interested, you can read about what it was I actually did wrong. So, first of all you need to know how the data whizzing backwards and forwards to a website like the one at snowbooks.com finds its way there. Much of the internet uses something called IP, which stands for Internet Protocol. It's basically a set of rules for packaging and routing data across a network. It involves taking the data you want to send, cutting it into roughly uniform chunks and putting them in the data equivalent of an envelope - or in internet parlance a 'packet'. Like any good envelope or packet it includes an address for where its contents are going and a return address saying where it's come from.
If you're trying to picture how digital data passing along a cable can be wrapped up into a packet I should explain that it's all just ones and zeroes, but the IP standard specifies a layout for those ones and zeroes. The first chunk of ones and zeroes will be the 'header' which is like the envelope. The header will contain the destination address and the return address and a few other pieces of information. Next will come the contents or 'payload' of the 'packet' i.e. the data you're trying to send across the internet, which could be something like a request to fetch the home page for the website at snowbooks.com.
But when it comes to the destination and return addresses, IP doesn't use a name like 'snowbooks.com'; it uses a number. This is similar to the way an old-style telephone doesn't let you type in the name of someone you want to talk to; it requires their phone number. So just like with telephones, the web requires a directory so that you can look up a name and get back a number that you can use to make a call. The way the directory is provided on the web is with something called 'DNS' and it's handled by your computer automatically so that you can surf the web without needing to know DNS even exists. But if it's your responsibility to add a new website to the web, you probably need to know the basics of what's going on.
The names on the web are 'domain names' (like 'snowbooks.com') and the numbers used to route data are called 'IP addresses' and most of them are four numbers, normally written separated by dots (e.g. '74.86.153.134'). You can buy the right to use a domain name by 'registering' it at one of the many Domain Name Registrars, like, say, nominet. But until that domain name has an IP address associated with it, nobody can use it to visit a website or send you e-mail.
So once you've registered yourself a domain name you need to create some DNS records (DNS actually stands for 'Domain Name System'). By 'record' what I mean is a single entry in the database of the world's DNS information. The simplest possible setup is just to create a single record that associates your domain name with the IP address of the computer running your website. That kind of record is called an 'A' record (A = Address) and you might write it out like this:
Domain name ; Type IP Address