Transcription
2/24/2012What Happens WhenYou Click Submit?A look at the protocols that make theweb go ‘round.Tech Software, Inc.Walden [email protected] do we care? Knowledge– More you know, better you are Performance– Too many requests make for a slow web page– CNN homepage is 158 requests with a cold cache! Reliability– Prevent failures Availability– Account for failures Security– Understand your cyber threats and attack surfaces1
2/24/2012Context SaaS Application Provider– IRBManager, SMART, Outcomes, Hibrid nginx – SSL offloadHAProxy – HA and load balancingIIS – Web serversASP.Net and ColdFusion MX – App ServersSQL Server – DatabaseEC2/AWS – Monitoring and DNSCommon Web ernet(BGP)SSL Offload(BGP)HTTPSHTTPDNSHA & LoadBalanceHTTPAWSHTTPHTTP HTTP QLSQLSQLDatabase2
2/24/2012Protocols in use HTTP – Hypertext Transfer ProtocolHTTPS – HTTP SSL (Secure Sockets Layer)DNS – Domain Name SystemSMTP – Simple Mail Transfer ProtocolBGP – Border Gateway ProtocolHTML – Hypertext Markup LanguageSQL – Structure Query LanguageTools Used Fiddler – Proxy server allowing capture andchange – fiddling – with request/response Firefox – Awesome browser Firebug – Firefox plugin showing details ofpage and request flow Wireshark – Network traffic capture andanalysis3
2/24/2012Request SMTPHTTP(S)InternetBrowser Event Model Where the magic startsModel defined by W3CEvent driven architectureClick / Submit triggers sequence of eventsEvents are Captured from the top downThen Bubbled from the bottom up“onclick” is end of capture, start of bubble4
2/24/2012Browser Event ModelJavaScript can change defaults JS can change the browser’s default behaviour onclick returning false cancels action (andbubble) preventDefault allows bubble, prevents action Replace default action with another:–––––Ajax request in place of full page postbackChange target URL before postbackAdd parm to link before requesting pageChange HTML on page, don’t contact server 5
2/24/2012Assuming no change Browser Performs one of two primary actions– GET – Request for resource from server– POST – Send of data to server Get’s response back and processes it– Interprets HTML and displays it– Requests additional resources as directed in HTML CSS, JS, JPG, Audio, Video, etc.– Runs JS in response to events on load, click, hover, change, blur, select, etc.Typical Request ServerResponseAnd then a miracle occurs6
2/24/2012Request Processing User enters http://www.cnn.comBrowser processes any JS on the pageBrowser “GET”s http://www.cnn.comTranslate www.cnn.com to an IP addressDNS – Domain Name SystemDomain Name System (DNS) Life without DNS? Need some new shoes? Come to– http://64.215.158.219 What the latest news? Just visit– http://157.166.226.25 OR– http://157.166.226.26 OR– http://157.166.225.18 OR– http://157.166.225.1907
2/24/2012Basics of DNS Queries and responses in 1 UDP packet Fast in and out, no session setup like TCP Servers can be caching, authoritative,referring, forwarding (recursive) or any combo Hardcoded list at the core (13 root servers) No DNS? You’re off the net– Redundant DNS servers– Multiple locations– Outsource itDNS Lookup1. www.cnn.com?Client2. Nope, go away!ServerAuthoritative, but not for cnn.com1. www.cnn.com?Client2. Nope, but try server at 199.22.3.70ServerReferring8
2/24/2012DNS Forwarding/Recursion1. www.cnn.com?ClientServer110. 157.166.255.18Server45. Go try Server 52. www.cnn.com?9. 157.166.255.18Server5Server34. www.cnn.com?8. 157.166.255.18Server23. www.cnn.com?DNS Performance 10 packets for that one lookup– 5 ms each? That’s 50 ms just for DNS lookup! Caching is very important!– But how long do we cache? That’s the TTL, orTime to Live DNS server location is important– “Closer” you are to DNS server, quicker theresponse– I’m 20ms to root L, 110ms to root K9
2/24/2012Request Processing I’ve got an IP, now what?– How do I get to 157.166.255.18? Single Default Gateway– That’s most consumer-companies Multiple paths to Internet?– Datacenters, ISPs, high-uptime requirements– Which one to take? What’s the “best” one? Border Gateway Protocol (BGP)– Core routing protocol for the Internet– Needed with multiple routes to/from the net– Tells outside world how to reach your IPsBGP “Route” Hong Kong to IRBManager in Westchester2914 is NTT Communications Global3356 is Level 311383 is Xand10
2/24/2012BGP Corruption Bad routing tables propagated across ‘net– Actually a file that exists on all core routers– Translates IP address ranges to ASNs Very rare, but can happen– Attempts to censor– Misconfiguration (or both) 2/08 Pakistan Telecom took YouTube off net– Court order for Pakistani residents– PT’s advertisement more specific (/24) than YouTube’s(/22)YouTube Availability Pakistan Telecom f-up, Feb 200811
2/24/2012One last BGP trick IP addresses “owned” by ASNs You list your ASN peers– Could you surface the same IP in multiple places?– 1.1.1.1 on a server in New York and Hong Kongand Bulgaria and India? Remember 13 root dns servers? Those are on250 actual servers throughout the world. http://www.root-servers.org/Request Processing Now I know where to send my packets, whatdo I send? Enter HTTP Text-based protocol, can use telnet! Request / Response style protocol Two main parts, the header and the body– NOT HTML head and body – First word of request is the verb– Not all verbs have bodies12
2/24/2012Request First word tells server what you want to do– GET / POST / HEAD / OPTIONS / etc. “Verbs” Rest of line identifies resource acting on– Index.html / logo.gif / styles.css / etc. Subsequent lines of request header provideadditional information– Caching, accepted languages, compression, UserAgent (what browser)– Blank line signified end of headerSample GETGET http://www.facebook.com/ HTTP/1.1Host: www.facebook.comUser-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:10.0.2)Gecko/20100101 Firefox/10.0.2Accept: text/html,application/xhtml xml,application/xml;q 0.9,*/*;q 0.8Accept-Language: en-us,en;q 0.5Accept-Encoding: gzip, deflateConnection: keep-aliveCookie: datr 5c1QSzbWMC3eCL7f9-wIPoo5Tw.;lu ggPNyFabb9lM5OG3wQer3IlA; c user 730972310;xs 785;presence 0CEchFDsubF 5b0Ac1 5dEp 5f730972310F7CC; p 3; act 1329780692760%2F4%3A2; wd 1838x94213
2/24/2012Sample POSTPOST https://irbmanager.becirb.com/index.cfm HTTP/1.1Host: irbmanager.becirb.comUser-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:10.0.2) Gecko/20100101 Firefox/10.0.2Accept: text/html,application/xhtml xml,application/xml;q 0.9,*/*;q 0.8Accept-Language: en-us,en;q 0.5Accept-Encoding: gzip, deflateConnection: keep-aliveReferer: https://irbmanager.becirb.com/Cookie: CFID 1625578; CFTOKEN 66144364; MACHINEID 9D259759-15C5-FC9B-F444-59BE715D886F;DBSESSIONGUID 83098DF0-D486-4D84-8B72-B5BA2C3A8D4A;utma 09.1;utmb 237304920.1.10.1329781309; utmc 237304920;utmz 237304920.1329781309.1.1.utmcsr (direct) utmccn (direct) utmcmd (none)Content-Type: application/x-www-form-urlencodedContent-Length: 116cm doLogin&SessionCookieValue 83098DF0-D486-4D84-8B72B5BA2C3A8D4A&UserName waldenl&Password xxxxxxxx&ClientID demo2Requests in flight Max of 2-8 requests per server “in flight” atany point in time Offload common data to secondary servers /domain names CDNs take load off primary servers– Also provide a second server name for requests Tradeoff of session startup vs. in-flighttransactions14
2/24/2012Response Server processes request and sends response– First line signifies success/failure (200, 302, 404)– Subsequent lines of header provide additionaldetails– Header ends with blank line Rest of response is the body of the response– Type depends on what was requested, HTML,MP3, JPG, etc.Response w/out BodyHTTP/1.1 302 Moved TemporarilyServer: nginx/1.0.11Date: Mon, 20 Feb 2012 23:52:10 GMTContent-Type: text/html; charset UTF-8Connection: keep-aliveCache-Control: privateLocation:Location /admin/index.cfmSet-Cookie: ASP.NET SessionId ;HttpOnly;expires Thu,01 Jan 1970 00:00:00 GMT;path /Set-Cookie: CLIENTID demo2;expires Wed, 12-Feb-204223:46:19 GMT;path /X-AspNet-Version: 4.0.30319X-Cluster-Server: ProdWeb1Content-Length: 015
2/24/2012Response w/BodyHTTP/1.1 200 OKConnection: closeDate: Mon, 20 Feb 2012 23:43:09 GMTContent-Type: text/html; charset UTF-8Content-Language: en-USServer: Microsoft-IIS/6.0X-Powered-By: ASP.NETVary: Accept-EncodingContent-Length: 44258 !DOCTYPE html html head title Find Protocol /title link rel "stylesheet" type "text/css" href "/includes/BECStyle.css" / !--[if IE 6] Cookies Request Header: Cookie: CFID 1625578; CFTOKEN 66144364; Response Header: Set-Cookie:CLIENTID demo2;expires Wed, 12-Feb-2042 23:46:19 GMT;path / Nothing more than a piece of data sent back andforth between client and server–––––ALWAYS constrained to the host the set themSession or time-based – When do they go awayHttpOnly – No access from JavaScriptSecure – Only sent to the server if securePath – Applies to the entire site, or a subdirectory16
2/24/2012TCP “handshake” 3-way handshake when starting a connection– Client - Server Packet w/SYN Flag– Server - Client Packet w/SYN and ACK Flag– Client - Server Packet w/ACK Flag– And now the session is established and we cansend HTTP Remember the 158 requests on CNN?– What if I’m 50ms from a server? 3x requests– That’s 23 seconds in session setup!Connection: keep-alive Client or server can request that theconnection be kept alive after each request– TCP connection used over and over– Eliminates the TCP setup time– Costs server (and client) resources17
2/24/2012Caching Many artifacts don’t change quickly Images, CSS files, Javascript, etc. These files are cached in the browser, in proxyservers and on the host infrastructure Caching improves performance at the expenseof currency. CNN homepage is 42 requests w/warm cache– 28 are ads, 6 are metricsRequest Processing We can send data back and forth But what about security?– Enter SSL – Secure Sockets Layer SSL established a “tunnel” through which yousend anything you want HTTPS is HTTP sent through an SSL tunnel18
2/24/2012SSL Tunnels Data is encrypted on one end and decryptedon the other Anyone viewing the data in the middle seesencrypted data HTTPS is HTTP SSL Tunnels SSL is also in SMTPS, SSH, FTPS, TelnetS, etc.Without SSLServerHacker BobPOST http://secure.buy.comCard 378734493671000&Exp 0116POST http://secure.buy.comCard 378734493671000&Exp 0116BobPOST http://secure.buy.comCard 378734493671000&Exp 011619
2/24/2012With SSLServerHacker BobSSLTunnelPOST http://secure.buy.comOrd 37&Card 378734493671000&Exp 0116POST http://secure.buy.comOrd 37&Card 378734493671000&Exp 0116BobSSLTunnelKeys and Ciphers Classic encryption is a simple substitution– A F, B G, C H etc.– Useful for DMCA and that’s about it Symmetric Key Encryption (SKE)– Same secret, or “key” is used on both ends– The password is “secret”– Pro: Fast– Con: “easy” to crack20
2/24/2012Keys and Ciphers Asymmetric Key Encryption (AKE)–––––Aka, Public Key / Private Key encryptionEncrypted with one key, decrypted with anotherKeys are created in a pair, one public, one privatePros: MUCH harder to crackCons: Computationally expensive Solution?!!!– Use AKE to encrypt a random session key to be usedfor encrypting the rest of the conversation with SKEHTTPS Setup New HTTP verb, CONNECT– Tells server that client wishes to go secure– Client sends server list of supported encryptions– Server responds with its PUBLIC key (certificate)– Client encrypts random data (symmetric key)using server’s public key and sends to server– Server and client now have a symmetric key– Server and client negotiate new randomsymmetric keys every so often21
2/24/2012HTTPS Pros and Cons Pros of HTTPS are obvious: Hacker Bob is unhappy! If you thought TCP handshake was bad – SSL handshake is 11 more packets! SKE is faster than AKE, but both are slower than noencryption Remember too, nothing can see inside an encryptedpacket, even when you want it to!– Load balancers, reverse proxies, data analysis We installed nginx precisely because of this– Need HAProxy to see traffic details for balancing andreprocessingDigital Signatures Encryption uses public key to encrypt But it works the other way too!– “Encrypt” the data with the private key– You can decrypt (verify) it with the public key Digital signatures are based on this fact SSL certs protect your security AND prove identity of server22
2/24/2012Request Processing Request is now completeBut what about that email?Simple Mail Transfer Protocol (SMTP)Another text-based protocol– Fun* to emulate with telnet– Simple request - response protocol– Like HTTP broken into a header and body with ablank line*I have a twisted definition of fun!SMTP Protocol Lookup “MX” record in DNSOpen TCP connection to port 25Issue HELO (Hello) or EHLO (Extended Helo)Send envelope from and toSend dataSay good byeAnd that’s why it’s so easy to SPAM!23
2/24/201224
2/24/2012What can go wrong? Aka: Why did we write our own SMTP service? Tracing and logging emails– We need to prove that a mail was delivered, or showwhy it wasn’t.– Misconfigured DNS (MX)– Full Mailboxes– Bad Email Addresses– Poor server responses (non-standard)– Incorrectly configured security settings on servers(expired certificates)Common Web )SSL Offload(BGP)HTTPSHTTPDNSHA & LoadBalanceHTTPAWSHTTP HTTP QLSQLSQLDatabase25
ASP.Netand ColdFusion MX - App Servers SQL Server - Database EC2/AWS - Monitoring and DNS Common Web Infrastructure Browser (HTML) Internet SSL Offload HA & Load Balance Web Web Web Database Background Tasks AWS DC Office SMTP DNS HTTP S HTTP HTTP HTTP HTTP HTTP S SQL SQL SQL SQL DNS DNS (BGP) (BGP) (BGP) (BGP) HTTP HTTP S .
