HTTP Header about on LoadBalance and Reverse Proxy environment

HTTP Header about on LoadBalance and Reverse Proxy environment
http://atman.memoab.com/articles/92


Summary

在负载均衡环境[LVS, LoadBalance设备]中有关HTTP Header的注意事项和在反向代理中的相关项.

Matter

用户体验在用户输入你的网址和按下回车的刹那即显效果,速度是不可或缺的因素之一. 本文中不会涉及到网络带宽 服务器架构或是css和javascripts的合并或分割. 只对工作中因负载均衡环境下影响和制约到包括网站页面本地缓存和过期时间的阐述, 我们知道多个HTTP请求和内容压缩将造成服务器CPU的上扬和进程/线程的消耗,但原则上只让用户与服务器群体传输尽量少的数据流,只传输更新的内容,且让静态数据贮存在用户本地.

在传输资料前,TCP做的SYN/ACK建立连接需要消耗大半的时间,

http1.jpg

所以我们可在后半用并发连接和持续连接在提速,表现在Apache 的配置方式可在httpd.conf中找到答案:
# prefork MPM
# StartServers: number of server processes to start
# MinSpareServers: minimum number of server processes which are kept spare
# MaxSpareServers: maximum number of server processes which are kept spare
# MaxClients: maximum number of server processes allowed to start
# MaxRequestsPerChild: maximum number of requests a server process serves

# worker MPM
# StartServers: initial number of server processes to start
# MaxClients: maximum number of simultaneous client connections
# MinSpareThreads: minimum number of worker threads which are kept spare
# MaxSpareThreads: maximum number of worker threads which are kept spare
# ThreadsPerChild: constant number of worker threads in each server process
# MaxRequestsPerChild: maximum number of requests a server process serves

# KeepAlive: Whether or not to allow persistent connections (more than
# one request per connection). Set to “Off” to deactivate.
#
KeepAlive On|Off

多道处理模块(MPM)中的预先派生进程比worker的线程与进程混合模式要稳定,但接受的请求海量大. 可在安装Apache Web使用configure脚本时用–with-mpm=worker来选择worker或者默认使用Prefork的预先派生进程模式.

一个Worker MPM的样例:

worker.PNG
#后一个为改善后的worker.c 的样式

持续连接在HTTP/1.0和HTTP 1.1中参数实现不同,而在RFC2616中建议一个连接对某服务器或反向代理最多两个持续连接,故避免在用户等待下载html时而Load 不出图片的方式是将图片交由另外的图片服务器来完成.

压缩带给数据传输的帮助是巨大的,在旧的Apache版本中多使用mod_gzip,现今在 Apache中已有mod_deflate代替. 浏览下概述:

概述

mod_deflate模块提供了DEFLATE输出过滤器,允许服务器在将输出内容发送到客户端以前进行压缩,以节约带宽。

大半用户使用的浏览器(IE,Firefox) 在使用HTTP Watch时你可查看到浏览器能接受的内容编码:

Accept-Encoding: gzip,deflate

我们的一个结果是让数据传输量少了3/4, 呵呵. 值得赞叹.

在你的配置文件中请注意提防那些“自以为是”的浏览器, 它们 不能很好的理解压缩数据.


# 插入过滤器
SetOutputFilter DEFLATE

# Netscape 4.x 有一些问题…
BrowserMatch ^Mozilla/4 gzip-only-text/html

# Netscape 4.06-4.08 有更多的问题
BrowserMatch ^Mozilla/4\.0[678] no-gzip

# MSIE 会伪装成 Netscape ,但是事实上它没有问题
BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
# 不压缩图片
SetEnvIfNoCase Request_URI \
\.(?:gif|jpe?g|png)$ no-gzip dont-vary

# 确保代理不会发送错误的内容
Header append Vary User-Agent env=!dont-vary

但在使用如Squid等反向代理中时注意附加Vary header对代理服务器的影响

mod_deflate模块发送一个”Vary: Accept-Encoding“HTTP应答头以提醒代理服务器:只对发送了正确”Accept-Encoding“头的客户端发送缓存的应答。这样可以防止不能正确处理压缩内容的浏览器接受到经过压缩的内容。

如果你按照某些特殊的条件拒绝了某些客户端的访问(比如User-Agent头),你必须手动配置一个额外的Vary头提醒代理服务器做额外的限制。比如,在一个典型的配置中的某处,如果额外的DEFLATE过滤器是否生效取决于User-Agent头,你应当在此处添加:

Header append Vary User-Agent

如果依照除请求头以外的其他条件决定是否使用压缩(例如:HTTP版本),你必须设置Vary头的值为”*“来完全阻止代理服务器的缓存。

示例

Header set Vary *

在Squid 2.6中对经过压缩的数据能Cache, 来提高 缓存命中率.
# TAG: cache_vary
# Set to off to disable caching of Vary:in objects.
#
#Default:
# cache_vary on

即当你的前端代理服务器实行加速WEB行为需要确认Cache_vary为开状态.

我们需要作些改变来合理使用Apache Modules, 对于频繁变动的页面内容我们可以在程序设置header来不缓存. Cal Henderson充分利用了Http 1.1规范的Last-Modified和ETag 来Cache内容.

3.11 Entity Tags

Entity tags are used for comparing two or more entities from the same requested resource. HTTP/1.1 uses entity tags in the ETag (section 14.19), If-Match (section 14.24), If-None-Match (section 14.26), and If-Range (section 14.27) header fields. The definition of how they are used and compared as cache validators is in section 13.3.3. An entity tag consists of an opaque quoted string, possibly prefixed by a weakness indicator.

entity-tag = [ weak ] opaque-tag
weak       = "W/"
opaque-tag = quoted-string

A “strong entity tag” MAY be shared by two entities of a resource only if they are equivalent by octet equality.

A “weak entity tag,” indicated by the “W/” prefix, MAY be shared by two entities of a resource only if the entities are equivalent and could be substituted for each other with no significant change in semantics. A weak entity tag can only be used for weak comparison.

An entity tag MUST be unique across all versions of all entities associated with a particular resource. A given entity tag value MAY be used for entities obtained by requests on different URIs. The use of the same entity tag value in conjunction with entities obtained by requests on different URIs does not imply the equivalence of those entities.

简单的说ETag Header是 文件修改时间、文件大小和inode号生成的校验和(checksum),那在多台服务器的负载均衡环境下会因部署内容的时间差异造成ETag的不同,故虽然请求同一个数据但会因不同机器的ETag而影响了响应. 具体表现为用户在第一次请求某一内容时下载而再次时浏览器会发现ETag不同而再次请求下载. [使用HTTP Watch查看,再次刷新时查看是否响应码为:304]

那可有对策? 嘿嘿, 试用下万能的Apache 核心指令FileEtag吧 .

top

FileETag 指令

说明 用以创建ETag应答头的文件的属性
语法 FileETag component
默认值 FileETag INode MTime Size
作用域 server config, virtual host, directory, .htaccess
覆盖项 FileInfo
状态 核心(C)
模块 core

FileETag指令配置了当文档是基于一个文件时用以创建ETag(实体标签)应答头的文件的属性(ETag的值用于进行缓冲管理以节约网络带宽)。在Apache1.3.22及以前,ETag的值总是由文件的inode(索引节点)、大小、最后修改时间决定。FileETag指令可以让您选择(如果您想进行选择)这其中哪些要素将被使用。主要关键字如下:

INode
文件的索引节点(inode)数
MTime
文件的最后修改日期及时间
Size
文件的字节数
All
所有存在的域,等价于:

FileETag INode MTime Size
None
如果一个文档是基于文件的,则不在应答中包含任何ETag

可以在INode, MTime, Size前加上”+“或”-“以改变由上层继承下来的默认值。任何没有上述前缀的关键字将立刻完全取消继承下来的设置。

如果一个目录的配置包含了”FileETag INode MTime Size“而其一个子目录包含了”FileETag -INode“那么这个子目录的设置(并会被其下任何没有进行覆盖的子目录继承)将等价于”FileETag MTime Size“。

OK, 因我们的不同服务器上的文件大小是一样的, 我们可以让创建的ETag基于数据大小.

FileETag MTime Size

这里不得不提的另一个机制Last-Modified.

  • 页面必须包含Last-Modified: 标记
    一般纯静态页面本身都会有Last-Modified信息,动态页面需要通过函数强制加上,比如在PHP中:
    // always modified now
    header(”Last-Modified: ” . gmdate(”D, d M Y H:i:s”) . ” GMT”);

我们用Apache mod_expires对text和image类型文件加有效期,控制应答时的Expires头内容和Cache-Control头的max-age指令. 这样客户端浏览器从缓存请求数据而不是服务器端. 当缓存中数据失效或过期,才决定从服务器更新数据.

示例:

# 启用有效期控制
ExpiresActive On
# GIF有效期为1个月
ExpiresByType image/gif A2592000
# HTML文档的有效期是最后修改时刻后的一星期
ExpiresByType text/html M604800

动态数据的过期定义如PHP可在程序行中加入:

//一个月过期
header("Expires: ".gmdate("D, d M Y H:i:s", time()+2592000)." GMT");
header("Cache-Control: max-age=2592000");

Resource

Squid Vary Header Support

Squid ETag Header Support

Cal Henderson 《Serving JavaScript Fast》Jennifer Vesperman Cache-Friendly Web Pages
HTTP Header 文档

62 Responses to “HTTP Header about on LoadBalance and Reverse Proxy environment ”


  1. 1 bhlfywaz cbsfmdgw Mar 28th, 2008 at 6:04 AM

    lscomg rghwsq djlviynk tlevd awuei rvamyibsw mvrhx

  2. 2 oqmfnbd fvjdmpt Mar 28th, 2008 at 6:05 AM

    dkhqjuvaf yfcliht foejkcsdv lqgbnfya hjygolzpb lmcob wcptbnro http://www.gwtvrxfsd.axdcbrvqe.com

  3. 3 viagra online Apr 11th, 2008 at 7:37 PM
  4. 4 Pen Pen Sin Pen mp3 Apr 18th, 2008 at 5:35 AM
  5. 5 Business May 6th, 2008 at 2:18 PM

    Nobody can stop me by posting these comments.
    http://www.businesswebhostings.net

  6. 6 Buy Viagra May 21st, 2008 at 10:41 PM
  7. 7 Vacation Packages May 26th, 2008 at 1:54 PM

    Vacation packages are always hard to search, but here are some options. http://www.imeem.com/people/Cc8kD5k

  8. 8 Just Like A Gillian Welch Song music May 27th, 2008 at 10:43 PM
  9. 9 ruhakbuhxwh May 31st, 2008 at 3:38 PM

    F1TQHv lhbxqtlkhbwj, [url=http://okapmgvgqibd.com/]okapmgvgqibd[/url], [link=http://ewgxdheyvsxd.com/]ewgxdheyvsxd[/link], http://afiaqgtjszfo.com/

  10. 10 ahizou Jun 1st, 2008 at 7:44 AM

    BBKKRr cuglfxlllebr, [url=http://thyyiahwzwey.com/]thyyiahwzwey[/url], [link=http://mdujnfcewbar.com/]mdujnfcewbar[/link], http://fsudbeposprj.com/

  11. 11 yjowaei Jun 1st, 2008 at 7:47 AM

    38vx10 draxpzxkksyq, [url=http://azhpiwxrwrsb.com/]azhpiwxrwrsb[/url], [link=http://uhtrdzfokgjv.com/]uhtrdzfokgjv[/link], http://jnrjgqpyqzie.com/

  12. 12 Gerasimos Jun 1st, 2008 at 11:32 AM

    It is a member of the site., symptoms of a fungal sinus infection, =-]],

  13. 13 Gondikas Jun 1st, 2008 at 12:21 PM

    Here is intresting people Lets talk!, motorcycle parts seattle, wyrhj,

  14. 14 Isaakios Jun 1st, 2008 at 1:12 PM

    Your site has very much liked me. I shall necessarily tell about him to the friends., four stroke scooter, tiqep,

  15. 15 Kostas Jun 1st, 2008 at 2:01 PM

    I would love to hear more about this , family poems graduation, :[[[,

  16. 16 Giorgos Jun 1st, 2008 at 2:50 PM

    Super-puper site!, luxury scooter, bhg,

  17. 17 Iacovos Jun 1st, 2008 at 3:39 PM

    The mission of the your site., full size scooter parts, mnd,

  18. 18 Glafkos Jun 1st, 2008 at 4:28 PM

    Your site is very convenient in navigation and has good design. Thanks!, dimetapp prednisone, yhtiy,

  19. 19 Konstantinos Jun 1st, 2008 at 5:17 PM

    hello, your site is lucky., elizabeth anania graduation photo, scsj,

  20. 20 Giannis Jun 1st, 2008 at 6:06 PM

    Very good site! I like it! Thanks!, syracuse university online mba, 37021,

  21. 21 Koinos Jun 1st, 2008 at 6:55 PM

    Your site is the best one!, syracuse university online mba, 8098,

  22. 22 Konstandinos Jun 1st, 2008 at 8:34 PM

    The mission of the your site., university of florida online business school, 8DDD,

  23. 23 Gerasimos Jun 1st, 2008 at 10:13 PM

    Keep up this great resource., free videos of women getting their pussys licked, 381,

  24. 24 Iakovos Jun 1st, 2008 at 11:03 PM

    Very good website you have here., freee lesbian movies big open pussies, 840,

  25. 25 Kosta Jun 1st, 2008 at 11:53 PM

    I would love to hear more about this , wear sunscreen graduation address, 50747,

  26. 26 Hristos Jun 2nd, 2008 at 12:41 AM

    The more you know about site, strawberry blonde pussys, 5152,

  27. 27 Giorgos Jun 2nd, 2008 at 1:32 AM
  28. 28 cyuyladhgqs Jun 2nd, 2008 at 11:49 AM

    tpCZYr hinzqjmguwzx, [url=http://jmqdyfvvflbg.com/]jmqdyfvvflbg[/url], [link=http://itrcgzgjihfr.com/]itrcgzgjihfr[/link], http://jdjlypmxvofs.com/

  29. 29 Free Articles Jun 2nd, 2008 at 6:04 PM

    Nice to play easy to win but hard to fight. http://www.adnarticles.com

  30. 30 Georgios Jun 3rd, 2008 at 12:50 AM

    Thanks for the interesting and informative site. Thats definitely what Ive been looking for., oxford university online mba, elkn,

  31. 31 Halu Jun 3rd, 2008 at 8:32 PM
  32. 32 Hippocrates Jun 3rd, 2008 at 9:37 PM

    Hello there,very nice place, retro drop leaf formica table, qft,

  33. 33 Kosmas Jun 4th, 2008 at 1:04 PM

    Very good website you have here., diet, :) ),

  34. 34 Alden Jun 4th, 2008 at 1:18 PM
  35. 35 Herakles Jun 4th, 2008 at 2:01 PM

    Your site has very much liked me. I shall necessarily tell about him to the friends., pussy, 93280,

  36. 36 Hermes Jun 4th, 2008 at 3:02 PM

    Hello there,very nice place, diet, hscm,

  37. 37 Kris Jun 4th, 2008 at 4:07 PM

    Your site has very much liked me. I shall necessarily tell about him to the friends., diet, >:DDD,

  38. 38 Free Articles Jun 4th, 2008 at 7:34 PM

    Nice to play and easy to win but hard to fight.
    http://www.adnarticles.com

  39. 39 salmonella Jun 4th, 2008 at 8:17 PM
  40. 40 pussy Jun 4th, 2008 at 10:57 PM

    YRjxSK pussy

  41. 41 Giatas Jun 5th, 2008 at 12:04 AM

    Your site has very much liked me. I shall necessarily tell about him to the friends., pussy, 8O O,

  42. 42 Iason Jun 5th, 2008 at 1:04 AM

    Very good site! I like it! Thanks!, pussy, =[[,

  43. 43 Gondikas Jun 5th, 2008 at 2:02 AM

    See you…Thanks, its great to receive such positive comments., diet, yjhvzg,

  44. 44 Gerasimos Jun 5th, 2008 at 3:43 AM

    You are the greatist!,

  45. 45 Kris Jun 5th, 2008 at 10:13 AM

    Thank you!Your site is very nice,

  46. 46 Grigoris Jun 5th, 2008 at 11:10 AM

    Very good site. Thank you!!!, complete blenderized diet, >:-D,

  47. 47 Ignatios Jun 5th, 2008 at 12:10 PM

    Save your time, please., twins pussy, fst,

  48. 48 rezko Jun 5th, 2008 at 1:14 PM
  49. 49 olympics2016 Jun 5th, 2008 at 2:21 PM
  50. 50 lizzie borden Jun 5th, 2008 at 4:19 PM
  51. 51 julianne hough Jun 5th, 2008 at 5:24 PM
  52. 52 gillian anderson Jun 5th, 2008 at 6:25 PM
  53. 53 hair loss Jun 6th, 2008 at 7:30 PM

    Nice work and I hope you will keep it up with your efforts. http://hairlosstreatmentx.com

  54. 54 lghgww Jun 7th, 2008 at 6:05 AM

    UpSY0o bzxbjetefspf, [url=http://tnwgsrhheiqq.com/]tnwgsrhheiqq[/url], [link=http://zwjuljoaxnlb.com/]zwjuljoaxnlb[/link], http://nzvqjqbewjxo.com/

  55. 55 Neo Jun 8th, 2008 at 2:34 AM
  56. 56 John Doe Jun 9th, 2008 at 10:27 AM

    3a429a56284df0a59716dc6243a6dfe9

  57. 57 Free Articles Jun 9th, 2008 at 3:56 PM

    Articles are always hard to search but here are some options.
    http://www.eshariq.com

  58. 58 Right Is Wrong mp3 download Jun 11th, 2008 at 1:03 AM
  59. 59 eo3b1pkloz Jun 12th, 2008 at 1:20 PM

    pnzaxcon6s t1fy5nyw5vu [URL=http://www.256979.com/1076203.html] ouaghs8wq02zkp0pm [/URL] ywaek2e84e9r

  60. 60 Declaration Jun 13th, 2008 at 8:55 PM
  61. 61 DesMoines Jun 15th, 2008 at 9:24 PM

    WX90af Treat all disasters as if they were trivialities but never treat a triviality as if it were a disaster.,

  1. 1 《Ourlinux》杂志 » Blog Archive » HTTP Header about on LoadBalance and Reverse Proxy environment Pingback on Jan 3rd, 2008 at 11:58 AM

Leave a Reply