Nginx Configuration for WPO

Two key configurations for Web performance optimization (WPO). In the future, we will look at how to tune TCP stack on nginx for better performance.

Turn on Gzip for text files
    gzip  on;
    gzip_comp_level 5;
    gzip_min_length  1000;
    gzip_types application/json text/css application/x-javascript text/javascript;

text/html is always compressed.

Add Expires header to static files
    location / {
        if ($request_uri ~* \.(ico|css|js|gif|jpe?g|png)$) {
                        expires 72h;
                        break;
        }
    }

    # This block will catch static file requests, such as images, css, js
    # The ?: prefix is a 'non-capturing' mark, meaning we do not require
    # the pattern to be captured into $1 which should help improve performance
    location ~* \.(?:ico|css|js|gif|jpe?g|png)$ {
        # Some basic cache-control for static files to be sent to the browser
        expires max;
        add_header Pragma public;
        add_header Cache-Control "public, must-revalidate, proxy-revalidate";
    }

    location ~* \.(ico|css|js|gif|jpe?g|png)(\?[0-9]+)?$ {
        expires max;
        break;
    }

Install pngcrush on mac

Steps to install pngcrush:
1.  (precondition for compile pngcrush source code) Installed Xcode, and installed Command Line Tools from Xcode -> Preferences -> Downloads -> Components -> Command Line Tools

2.  Download pngcrush latest version
http://sourceforge.net/projects/pmt/files/pngcrush/

3.  Extract the downloaded package and run "make"

4.  Move or copy pngcrush to one $PATH folder
sudo cp pngcrush /usr/sbin/

5.  Try out
pngcrush -reduce -brute original-image.png new-image.png

Quick notes about image optimization:
When we talk about image optimization for web performance, we usually consider use pngcrush to optimize PNG file, and use Jpegtrans to optimize JPEG file. Usually we automate this in web build. I once wrote a post regarding this topic.

On mac, there is an app called ImageOptim which supports many cool tools.

https://www.xebew.com/xebinders/#join/BlGnxLjLT6f7cOiJgcpCbTC

Image Optimization

Recently we did pngout for more than 2000 files, but few of them could not be optimized due to bit depth is 64 or null. However, we have not covered other image types like JPEG, GIF. There are a lot of discussions and talks about image optimization for web performance. In our project so far, we just started the first step, but we followed 80/20 rule to optimize our website.

Generally speaking, image optimization has 2 categories: lossless and lossy.

• Lossless Image Compression: Remove surplus info from image (Smush.it)
• Lossy Image Compression: Match image quality to screen resolution

The safe way and easy way is to do lossless image optimization during build process and development phase. There are some high level guidelines for this optimization effort:

• Don't use GIF unless you need animated GIF
• Crush PNG (pngout)
• Crush JPG (jpegtran)
• Know PNG24 to PNG8 Conversion Guidelines
• Know the time to choose JPG over PNG (Don't use PNG for everything)
• Combine images (csssprite/spriteMe)
• Compress favicon
• Crop and compress user uploaded avatar

PNG or JPG?

• PNG is a lossless compression format
• JPG doesn't support transparency
• PNG works best for vector type graphics with hard lines. 
• JPG works best for anything with complex gradients like photo, screenshot etc
• Use imagemagick to batch convert PNG to JPG

For more info about PNG vs JPG, check out PNG vs JPG: 6 simple lessons you can learn from our mistakes

Optimize PNG file

• color depth
  Color depth can range from 1 to 64 bits per pixel, reduce color depth can reduce file size

• ancillary chunks
  PNG supports metadata—remove it can reduce file size

• interlacing
  As each pass of the Adam7 algorithm is separately filtered, this can increase file size

• filter
  The choice of filter is potentially very variable and impacting file size

• compression
  DEFLATE compressors can produce smaller files.

As some tools are PNG-specific, while others only optimize DEFLATE, in general one must use a combination of 2 tools in sequence for optimal compression: one which optimizes filters (and removes ancillary chunks), and one which optimizes DEFLATE. Most commonly, OptiPNG is used for the first (non-DEFLATE) step, and either of AdvanceCOMP or PNGOUT is used for the DEFLATE step. For more info or detailed comparison among PNG, JPEG, TIFF, check out http://en.wikipedia.org/wiki/Portable_Network_Graphics

Frequently used Tools
pngcrush
pngout
OptiPNG
ImageOptim
imgopt
Smush.it

CSS performance

Recently we are looking at Javascript performance, and there are many good resources (like High Performance Javascript book), when looking at CSS, my first thought is all about selector. However, after reading http://www.slideshare.net/booshtukka/high-performance-css slide, I realize that apart from selector/rendering performance, size/request numbers are still WPO fundamentals from CSS performance standpoint. Here are reading notes about "High Performance CSS" deck.

Size
name - using abbreviation
shortcuts (zeror no unit, decimal beginning with zero need no zero, shorthand)
no final semicolon
minify (use a build script to run YUI compressor)
run ImageOptim against images
Gzip CSS

Number of requests
Concatenate your files
Use CSS sprites for buttons/hover status

Rendering/selector efficiency
from right to left (make key selector efficient)
avoid being too specific (avoid overqualified selectors)
universal selector selects every element, breaks inheritance
hardware acceleration - not suggested

Other tips
avoid CSS greater than 20KB
avoid CSS holds more than 4096 rules
border-radius, box-shadow and RGBA are all slow

Understanding about performance optimization

I am working on performance optimization task for a web application for few months, here I am trying to summarize my understanding about performance optimization.

1. Performance optimization needs culture
• Coding Honor says: Performance is a feature
• Fred Wilson's 10 Golden Principles say: speed is more than a feature
• Design with performance in mind
2. Performance optimization needs tools and guidelines
• High Performance Web Site (Steve Sounders)
• Yahoo Best Practices for Speeding Up Your Web Site
• Lots of tools from WPO companies and open source communities
3. Performance optimization needs 80/20 rule
• Find top issues and fix them
• Put small things on hold
4. Performance optimization needs data
• Measurement (problem) -> Analysis (root cause) -> Optimization (solution) -> Measurement (new problem)
• Google says: Every millisecond counts
5. Performance optimization needs passion
• Do extensive research
• Repeat the optimization loop (It is a heuristic loop instead of dead loop)
• Evangelist performance optimization
• Make faster web and better life

strangeloop cyber Monday tips

Recently I got a series of emails with tips from strangeloop regarding web site optimization before cyber Monday. Each tip has the core basic about WPO, and also introduces their Site Optimizer. Here I want to summarize the email series and their optimizer.

In my mind, WPO always starts with measurement (testing) to understand the performance problem, then find tools, solutions to solve the problem, then measure again. This is an iteration process, and focus on top 3 issues in each iteration will make your WPO more efficient. I blogged my understanding of core practices or Fundamentals of WPO 2 month ago. From  strangeloop tips, another key word I find very suitable is "simplify". What we are trying to achieve is to simplify the web site, less http request and less data.

Let's look at the tips from strangeloop.
http://www.strangeloopnetworks.com/cyber-monday-2-weeks/

1. Test your site, using tools (webpagetest.org)
2. Make sure your site follows core best practices
   
   • Text compression: The easiest way to reduce your page sizes 
   • Keep-alives: Control your TCP connections
3. Identify and fix your sluggish third-party content
   
   • Audit your 3rd-party content
   • Use the latest version of your widgets
   • load asynchronously wherever possible

Then let's look at how Site Optimizer works
http://www.strangeloopnetworks.com/products/overview/how-it-works/

1. Simplify the page
2. Recognize different browsers
3. Preload relevant page elements
4. Optimize for repeat visits and flows
5. Start loading fasterOptimize third-party content

Fundamentals of WPO

There are two fundamental factors (action items) for Web Performance Optimization (WPO), request number and data size. There are top rules and plenty of tools to assist these two optimization. This is common sense and very straightforward to understand the importance. One metaphor about moving in real life, if you can throw away what you don't necessarily need, and keep your belongs to minimum say 1 U-haul truck can hold, then you can make the move fast because you only need 1 round trip with minimum stuff.


Reduce Http request - Send it as infrequently as possible
Http round trip is expensive esp. with a long RTT, or for new http connection. Reducing http request can be achieved using the following:

• Merge static resources (CSS, JS, Image)
• Add Cache control (including ajax?)
• Combine dynamic requests
• Build Single page application (using Ajax)
• Avoid redirect
• Fewer DNS lookup
  

Reduce download size - Send as little data as possible
Small data size saves bandwidth (tcp package numbers). It can be achieved using below ways:

• Gzip resources
• Minify JS and CSS (Code optimization, obfuscation, duplication removal)
• Crush images
• Add Cache control (caching in browser)
• HTML5 local storage, local cache

For more info, refer to 14 Rules for Faster-Loading Web Sites

DNS Override

DNS server keeps website domain name and its IP address mapping. Public DNS servers are supposed to keep in sync (eventually consistence), so every browser (user agent) can do DNS lookup using different DNS servers, and get same IP address for same domain name.
Domain sharing is a technique to optimize web performance for parallel downloading, but it brings more DNS lookup effort on browser side. The lookup sequence is described below, and browse stops if found corresponding IP address, then loads the website.

1. Browser cache
2. Computer hosts file for a DNS entry (OS dns cache?)
3. Default DNS server (this is usually ISP's or your employer's DNS server)
4. Other DNS servers
5. If domain name can not be resolved, browser will display "server not found" error page

DNS override is to change domain name to IP address mapping in step 2 or step3, so that local computer will point to another IP address for the same domain, but public users still connect to existing IP address for the same domain name.

Override DNS entry using local DNS server
In step 3, it is usually done in internal DNS server or local DNS server which is used internally in your corporate, and it will impact all employees. Change ISP's DNS sever is actually changing all public DNS servers, so override should not happen on this DNS server.

Override DNS entry using hosts file
It is kind of key-value pair mapping file for domain name and IP address. Here is the list of file path on different operating systems.

1. Windows: C:\Windows\System32\etc\drivers\hosts
2. Linux:  /etc/hosts
3. Mac: /etc/hosts 

DNS Suffix
When you ipconfig /all, it will print out details about your network connections. Two more entries about DNS are Primary Dns Suffix (usually it is your computer registered domain) and DNS Suffix Search List (which is a list of your domains suffix when you connect using partial of FQDN).

For example, if the ipconfig /all output looks like below:

C:\Documents and Settings\mypc>ipconfig /all

Windows IP Configuration

        Host Name . . . . . . . . . . . . : mypc23434
        Primary Dns Suffix  . . . . . . . : example.local
        Node Type . . . . . . . . . . . . : Hybrid
        IP Routing Enabled. . . . . . . . : No
        WINS Proxy Enabled. . . . . . . . : No
        DNS Suffix Search List. . . . . . : example.local
                                            corp.example.com
                                            new-example.com

If you have some FQDN like test.example.local, exam.corp.example.com, motor.new-example.com, then if you ping using short name test, exam or motor, you will get DNS suffix appended to find the FQDN.

Summary about High Performance Mobile Meetup

This Tuesday (Aug 30, 2011) I attended the SF web performance meetup at LinkedIn (Mountain View). Steve Souders presented a great talk "High Performance Mobile". Apart from this year Velocity conference HttpArchive lighting demo, this is the second time I joined his session. I read twice his previous talk slide "High Performance HTML5" at SF performance meet up, but could not join in person. Of course, I borrowed and read his two famous books.

The event started at 6:30pm, welcomed attendees with portable plastic water bottle and Mexican food. Steve began his talk around 7pm after some introduction from meetup organizer (Aaron Kulick) and LinkedIn performance lead (they are hiring performance engineers). The talk consisted of 4 parts, you can find details from http://www.slideshare.net/souders/high-performance-mobile-sfsv-web-perf

Part 1: WPO
Nothing new, but he reiterated the importance and role of WPO. The Web Is Dead (http://www.wired.com/magazine/2010/08/ff_webrip/all/1), will this be true and what is the fate of WPO? It might be too early to conclude "The web is dead". One thing is true, the web is evolving with HTML5.

Benefits of WPO

1. drives traffic
2. improves UX
3. increases revenue
4. reduces costs

Part 2: Why Mobile
Data and analysis show that Mobile is very important, but slow. Also the "Road is not clear" for performance optimization

Part 3: Mobile Best practices
Most desktop Web performance rules still apply. Steve mainly shared 5 items to which he thinks Web performance engineers need pay more attention.

1. Reduce Http request (sprite/dataURI/CSS3/Canvas) - this should be the golden rule
2. Responsive images (sencha.io src/DeviceAtlas/adaptive-images.com) - previously read an article tweeted by Stoyan, have some general idea about this
3. script async & defer (execute when available, execute when parsing finished. He also mentioned his controlJS) - heard this many times, are all new browsers supporting them? should we include javascript using async & defer
4. Appcache (5M+ limit) - from HTML5
5. Local storage (window.localStorage) - from HTML5

Part 4: Mobile tools
The impressive one of this part was his demo after he briefed following 4 tools. Steve also mentioned his bookmarklet for mobile performance but he didn't add into his deck.

1. pacpperf
2. jdrop
3. blaze.io
4. weinre (WEb INspector REmote)

After that, there was book giveaway and e-book lottery. Somehow I didn't have a number, so left before venue was clear up. (I hope I can get a signed copy of his book)

Key takeaways

1. Mobile is important but very slow
2. There are challenges to make mobile fast
3. There are tools to assist mobile performance
4. Mobile winners will be fast

Thanks Steve/Aaron/Sponsors to make this meetup so successful. Looking forward to next meetup in south bay area.

Different Testings

Testing is an art than science. There are different testings for different purposes to ensure the software quality. The goal is same, but the process, strategy and methodology are different from different testings.

Correctness testing
Correctness is the minimum requirement of software, the essential purpose of testing. It is for software quality, also called function testing.

Black-box testing
test data are derived from the specified functional requirements without regard to the final program structure. It is also termed data-driven, input/output driven, or requirements-based testing.

White-box testing
the structure and flow of the software under test are visible to the tester.

Performance testing
a process that focuses on testing individual components of the web app, such as databases, algorithms, network infrastructure, and cache layers under certain load

Load testing
a process of determining how an application under specific volumes of load, usually a range of the upper and lower limits expected by the business. Endurance testing is also part of this testing type.

Stress testing
a process of identifying when and how systems fail (and recover) under extreme levels of load. Also known as negative testing or destructive testing.

Reliability testing
a process of finding the probability of failure-free operation of a system.

Security testing
identifying and removing software flaws that may potentially lead to security violations, and validating the effectiveness of security measures. Simulated security attacks can be performed to find vulnerabilities.

I summarized above testings based on below excellent blogs and paper. It is interesting to understand these terms better while working with QA (quality assurance) team in daily work.
http://agiletesting.blogspot.com/2005/02/performance-vs-load-vs-stress-testing.html
http://agiletesting.blogspot.com/2005/04/more-on-performance-vs-load-testing.html
http://blog.browsermob.com/2008/12/performance-vs-load-vs-stress-testing/
http://www.ece.cmu.edu/~koopman/des_s99/sw_testing/

Website is not an island

Today I joined the Webinar from compuware talking about How to Mitigate the performance risk of 3rd party Web components. Apart from their impressive application delivery chain explanation of the necessity to test and monitor Web performance from end user perspective, "No website is an island" is also an interesting topic with regard to the importance of looking at third party web component performance.

There are quite a few talks about Ad and Google Analytics performance. Current Web sites are getting richer (average page size is keeping increasing). Most contents are not served from single host (domain), instead from different outside sources, like Advertisement, Analytics, NewsFeed, Blog, Social network, APIs, Video, Shopping cart, Search engine, Rating&Review and Cloud objects. The benefits of using 3rd party components outweigh the risks - it is not under your control.

The best way is to mitigate the risk using below guidelines:

• Choose 3rd party component with high SLA
• Decide on a mitigation strategy (fault-tolerance, fallback, design for failure, alternative)
• Test it under all conditions
• Figure out a way to monitor it

During the webinar, Compuware CTO also talked about 5 best practices, and I totally agree the tips for Mobile Web site because it covers 3 major factors (# of requests, download size and network latency) regarding mobile Web performance.

• Minimize third-party content
• Limit # of hosts, connections and requests
• Keep size small and use CDN

Web browsers and engines

When programming Web, we need consider cross browsers to make sure the HTML/CSS/Javascript codes work well in different Web browsers. As of today, the most popular web browsers are Internet Explorer, Firefox, Google Chrome, Safari, and Opera.
http://en.wikipedia.org/wiki/Web_browser

Web browser has an engine to take marked up content (such as HTML, XML, image files, etc.) and formatting information (such as CSS, XSL, etc.) and displays the formatted content on the screen. (for data and view) This engine is also called sometimes called layout engine or rendering engine.
http://en.wikipedia.org/wiki/Web_browser_engine

1. Internet Explorer 9 uses Trident
2. Firefox uses Gecko
3. Chrome uses WebKit
4. Safari uses WebKit
5. Opera uses Presto

http://en.wikipedia.org/wiki/Comparison_of_layout_engines_(HTML)

Apart from layout engine, Web browser also needs a javascript engine to execute javascript (for actions).  Most new javascript engines have JIT compiler for better performance (See SunSpider for benchmark data)

1. Chakra: A new IE JScript engine used in Internet Explorer 9. It was first previewed at MIX 10 as part of the Internet Explorer Platform Preview.
2. SpiderMonkey: A JavaScript engine in Mozilla Gecko applications, including Firefox. The engine has two types of JIT compilers, that are sometimes referred to as JägerMonkey or TraceMonkey.
3. V8: A JavaScript engine used in Google Chrome.
4. SquirrelFish: The JavaScript engine for Apple Inc.'s WebKit. Also known as Nitro.(JavascriptCore)
5. Carakan: A JavaScript engine developed by Opera Software ASA, included in the 10.50 release of the Opera web browser.

http://en.wikipedia.org/wiki/List_of_ECMAScript_engines

In my mind, new Web browsers are getting faster, and Web development is more focusing on framework, pattern and productivity. We are moving to HTML5, Javascript (latest version 1.8.5) and CSS3, and expecting to see less difference across these browsers. A new Web era is coming.

Cache related Http Headers

Http headers can instruct what kind of cache mechanism browser and proxy should obey along the request/response chain. For static resources (Javascript, CSS, images, flash etc), it is suggested to apply cache on browser or proxy side to reduce # of Http requests.

Response Headers:
Cache-Control     Tells all caching mechanisms from server to client whether they may cache this object (public/private to control if browser or proxy cache, no-store to control if save to disk, max-age is to control how long to cache)
Expires     Gives the date/time after which the response is considered stale (suggested 1 year from now, for aggressive static resource cache)
Date     The date and time that the message was sent (It is useful for Expires by date time)
ETag     An identifier for a specific version of a resource, often a message digest (Suggest to disable it for performance, or re-configure ETag to remove server specific info. See If-None-Match. ETag takes precedence over Last-Modified if both exist)
Last-Modified     The last modified date for the requested object, in RFC 2822 format (for conditional get, 304 Not Modified, see If-Modified-Since)
Pragma     Implementation-specific headers that may have various effects anywhere along the request-response chain. (Http1.0, example is Pragma: no-cache)
Vary     Tells downstream proxies how to match future request headers to decide whether the cached response can be used rather than requesting a fresh one from the origin server. (The most common case is to set Vary: Accept-Encoding, so that proxy knows if return cached compressed data to browser)

Request Headers:
Cache-Control     Used to specify directives that MUST be obeyed by all caching mechanisms along the request/response chain
If-Modified-Since     Allows a 304 Not Modified to be returned if content is unchanged
If-None-Match     Allows a 304 Not Modified to be returned if content is unchanged
Pragma     Implementation-specific headers that may have various effects anywhere along the request-response chain

Recommendations:
It is important to specify one of Expires or Cache-Control max-age, and one of Last-Modified or ETag, for all cacheable resources. It is redundant to specify both Expires and Cache-Control: max-age, or to specify both Last-Modified and ETag.

You use the Cache-control: public header to indicate that a resource can be cached by public web proxies in addition to the browser that issued the request.

Avoiding caching
HTTP version 1.1 -> Cache-Control: no-cache
HTTP version 1.0 -> Setting the Expires  header field value to a time earlier than the response time

Reference
http://tools.ietf.org/html/rfc2616
http://en.wikipedia.org/wiki/List_of_HTTP_header_fields
http://code.google.com/speed/page-speed/docs/caching.html#LeverageBrowserCaching
http://code.google.com/p/doctype/wiki/ArticleHttpCaching

Mobile Web Optimization Webinar Takeaway

Last week I attended a webinar from Compuware talking about mobile web optimization using page speed. The talk was very clear and well organized. It first went through the importance of mobile web performance, then discussed key difference between mobile and desktop, then detailed page speed rules about mobile web.

Browser is the entry point to mobile web
The browser is becoming the integration platform
The browser is becoming more complex
    - # of hosts per user transaction
    - Many RIA frameworks
    - Performance differences across devices

Free performance tools
Page Speed
WebPagetest
dynaTrace Ajax Edition

Mobile web page load process
Mobile channel establishment
DNS lookup
TCP connect
Http request
Parse & Layout (subrequests)

Key differences between mobile and desktop
Networks:
    round-trip time (High channel establishment time, lower RTT)
    bandwidth (3G vs Cable)
Devices:
    CPU (JS execution times, layout times, 10x JS runtime cost, 1 ms per kb parsing)
    memory (more code/objects - more GC, more DOM, more memory)
Interaction model
    (touch vs click, mobile click event with 300-500ms delay)

Page Speed Rules
Use an application cache (how about localstorage?)
Defer JS parsing (how about deferring JS download?)
make landing page redirects cacheable (Cache-control: private, max-age > 0) (How long? 301 and 302 are both cacheable?)
prefer touch events (why not disable click event on mobile?)

Reference:
http://slidesha.re/qgC8n3
http://www.slideshare.net/Gomez_Inc/optimizing-web-and-mobile-site-performance-using-page-speed

90% line in JMeter aggregate report

I considered 90% line is the response data 90% users will see, but recently realized this was wrong. JMeter online Help has detailed explanation about every item in Aggregate report.

The 90% line tells you that 90% of the samples fell at or below that number. However, it is more meaningful than average in terms of SLA. We expect it within 2x of average time. That is, if average time is 500ms, we expect 90% line is less than 1000ms. Otherwise the system fluctuates a lot.

Web Performance Sites

http://www.webperformancetoday.com/
http://www.perfplanet.com/
http://stevesouders.com/

Top methods for Faster, Speedier web sites
http://www.askapache.com/web-cache/top-methods-for-faster-speedier-web-sites.html

BetterExplained Learn Right, Not Rote.
http://betterexplained.com/

http://www.tutkiun.com/2010/07/load-javascript-after-pageload.html
http://www.codeproject.com/KB/aspnet/CombineAndMinify.aspx
http://dean.edwards.name/packer/
http://www.jslint.com/

http://www.sitepoint.com/article/web-site-optimization-steps
http://livehttpheaders.mozdev.org/
http://yuiblog.com/blog/2007/01/04/performance-research-part-2/

• http://jsbeautifier.org/
• Microsoft ajax minifier – The Microsoft Ajax Minifier enables you to improve the performance of your Ajax applications by reducing the size of your Cascading Style Sheet and JavaScript files (which also provides the hypercrunch option to make it less readable)
• http://javascriptobfuscator.com/default.aspx – This uses obfuscation techniques (shown in image above!). Nice.
• There is also the YUI JS Compressor – http://developer.yahoo.com/yui/compressor/
• http://dean.edwards.name/packer/
• encode base 64 – http://www.shell-tools.net/index.php?op=base64_enc
• decode base 64 – http://www.shell-tools.net/index.php?op=base64_dec