2010这一年

Posted on by
Reply

每次写我的这一年都会感慨时间之快，感觉去年元旦就在昨天，日历却即将翻过整整一年。

这一年，是人生一次重大转变的第一年，些许的不适应。
这一年，多了一些焦虑，三四月的痛苦还历历在目，幸运的是我选择了坚持。
这一年，多了些失眠的日子，这些只有自己明白。
这一年，少了年轻的偏执，怀念那种为一个目标而奋斗的偏执。多了些淡定，告诉自己三十的困扰不过是暂时的，淡定和坚持的为心中的理想奋斗就会收获。
这一年，第一次出了国。
这一年，父母第一次来了北京。
这一年，借钱在家买了房子，父母终于没了拆迁的烦恼。
这一年，及时的还掉了借款。感谢几位。
这一年，西安的房子好像没怎么涨，继续出租吧。
这一年，北京的房价继续涨，房东也给涨了10％。
这一年，大哥终于把他家房子装修完毕，我真佩服他的拖劲。
这一年，大姐的工作还算稳定，注意身体。毛毛你要看到你mama的付出。
这一年，二哥也有了新的发展方向，希望你坚持，活跃的思想加坚持定会成功。看好你。
这一年，爸爸妈妈身体还好，这是我最担心的。妈妈的膝盖要保养好。爸爸保养好心脑。你们健康是儿女的幸福也是你们的幸福。
这一年，去媳妇家的次数少了些，不过元旦就去了。
这一年，媳妇继续的支持着我，帮助着我。想说谢谢你，老婆注意身体。
这一年，老婆工作得到领导赏识，嗯不错。最重要的涨工资啦。
这一年，不太敢看校友录，铺天盖地孩子照片。明年我是不是也得加速了。
这一年，好友从北京到底回了家乡，还带了个媳妇回去。不要忘了我，祝你事业家庭双丰收。
这一年，有同事选择了新职业，我相信，认真的选择的结果一定是正确的，犹如去年我的选择。祝新年事业展宏图。顶你。
这一年，小辈们成绩和表现都不错，表扬下文杰同学。
这一年，博客换成了Wordpress，选择是对的，可以手机 ipad客户端随时登录，感慨3G时代。此文就是用手机发的。
这一年，英语有所长进，还需新年更努力。
这一年，学到了很多，技术的，人的。
这一年，成熟了许多。
这一年，是坚持的 焦虑的 收获的 有点累的 有压力的 成长的。
2010我说努力，释放压力，解决嘴巴英语，看来后两者我没做到。2011，我说更努力，完成既定目标。

－－－北漂的80后，非富二代。

Posted with WordPress for BlackBerry.

Simulating WAN network delay

Posted on by
Reply

Simulating WAN network delay

Motivation

When testing the version of AutoFS from RHEL-3, Update 4 in a global WAN environment I discovered an interesting bug. When logged into a client machine in London, every now & attempts to access a mount from an NFS server in Boston would fail perhaps 25% of the time. The server was clearly online, since mounts from other clients would work fine, and there were no obvious errors in the logs from either the client or server. After a little more testing I discovered that mounts between London and LA would fail 100% reliably [sic]. This led me to believe that there was some part of the AutoFS mount process which was sensitive to network round trip time. Sure enough, there was a piece of code which checked for ‘livliness’ of the server by sending an RPC ping and had a fixed timeout of 100ms. Well on my particuarly WAN, ping times between London and Boston were 100ms +/- 5ms – no wonder it failed seemingly at random. Once identified the bug itself was trivially fixable by letting the code fallback to a longer timeout, if no server being tested had replied within the short timeout. I then got on to thinking about how you might build a system under which AutoFS could be reliably tested in a lab, without requiring co-location of part of the system half-way around the globe.

Planned solution

The obvious solution is to figure out a way to introduce arbitrary network packet delay between two hosts on the same subnet. I considered two possibilities

  1. Setup a virtual interface on one of the hosts using the tun driver, and have the usre space daemon processing it queue up packets for Xms before sending them out. A few routing table entries and IPtables rules could then be used to redirect traffic on eth0 via the take tun0 interface.
  2. Use the IPtables QUEUE target to intercept traffic on the actual network interface, redirecting to a userspace program to delay them

In the end I chose the second one since it seem to potentially require less work to implement and setup.

Implementation

CPAN has a Perl module IPQueue.pm which backends onto the libipq.so library which is part of IPTable codebase. With this, the Perl userspace daemon becomes obscenely easy to write. In pseudo-code

forever
  foreach queued packet
     if queue time > requested delay
        accept packet

  wait for an incoming packet

  add packet to queue

The actual code is in the script delay-net.pl.

Running it

The first task is to load up the neccessary iptables kernel modules

modprobe iptable_filter
modprobe ip_queue
modprobe ipt_ttl

Now start the daemon – its important we do this before adding the IPTables rules to QUEUE traffic, otherwise you’ll potentially lock yourself out of the machine! It takes the number of milliseconds delay its only command line argument, so lets delay for 300 milliseconds

./delay-net.pl 300

Then add a rule to redirect incoming traffic from either the entire network, or better, from a particular host.

iptables -A INPUT --source 192.168.16.4 -j QUEUE

If you now run ‘ping’ from the host mentioned in the iptables rule, you should see an nice (approximate) 300ms delay.

Downloads

For convenience here are downloads of the various packages required to run on a RHEL-3 system

Studying Normal Traffic, Part Three: TCP Headers

Posted on by
Reply

Studying Normal Traffic, Part Three: TCP Headers

Anonymous's picture
13 May 2001

by Karen Kent Frederick

Studying Normal Traffic, Part Three: TCP Headers
by Karen Kent Frederick
last updated May 14, 2001
This is the final article in a three-part series devoted to studying normal traffic. As was explained in Studying Normal Traffic, Part One and Studying Normal Traffic,Part Two: Studying FTP Traffic, we often focus our attention on the characteristics of suspicious packets without first becoming familiar with the characteristics of normal traffic. A good and easy way of doing this is to generate, capture and examine your own normal traffic. The first two articles in this series showed how to capture packets using WinDump and reviewed some of the basics of normal TCP/IP traffic. In this article, we will be looking at two other aspects of normal TCP traffic: the structure of TCP packets, and the use of TCP options. Note that in order to understand this material, you should already know the fundamentals of TCP/IP.

TCP Packet Structure

If you’ve read the previous articles in this series or worked with tcpdump or WinDump, you are probably familiar with the format of their output for TCP traffic. If not, here’s a quick review of the meaning of the fields:

13:10:30.134481 [timestamp] my.win98.box.2172 [source address and port ] > anon.ftp.box.21: [destination address and port] S [TCP flags] 71870464:71870464 [sequence numbers] (0) [bytes of data] win 65535 [TCP window size] [TCP options] (DF) [don't fragment flag is set] (ttl 128, id 44644) [time to live value, and IP identification number]

Let’s start by looking at an example that shows the first packet (SYN) of a TCP three-way handshake. It was initiated by a Windows 98 computer and the destination machine is an anonymous FTP server.

13:10:30.134481 my.win98.box.2172 > anon.ftp.box.21: S 71870464:71870464(0) win 65535  (DF) (ttl 128, id 44644)

Most of the fields shown in this entry have been discussed in the previous articles in this series. One field that we haven’t discussed at all are the TCP options; in this example, they are: . So what do they mean, and are they normal or not?

Before we can answer that, we need to know more about the structure of TCP packets themselves. The log entry shown above is actually a text representation of the values of some of the bits and bytes in the packet; the log snippet below shows the actual values of the entire packet in hex. Each pair of numbers (such as “45″) constitutes a byte. You can view packets in this way by running tcpdump or WinDump with the -x option.

 4500 0030 ae17 4000 8006 f398 xxxx xxxx yyyy yyyy 0c37 0015 0526 a98f 0000 0000 7002 ffff 6f06 0000 0204 05b4 0101 0402

The first part of the packet, shown here in italics, is the IP header. It contains several values, including the source and destination IP addresses, the IP identification number, the type of service (TOS), fragmentation information, and the time to live (TTL). The basic IP header is 20 bytes long; if IP options were present, they would add additional bytes to the header. Examples of IP options include loose source routing, strict source routing, timestamps and record route.

Because the use of IP options changes the length of the IP header, a value is set in the header to indicate how many bytes long the header is. It’s important to note that the length of the IP header must be a multiple of 4 bytes. In the example shown above, the second hex digit is 5; this indicates that the length of the IP header is five 4-byte chunks, or 20 bytes total. Since this is the minimum length of the IP header, this tells us that no IP options have been set.

An IP packet is composed of the IP header (with or without options) and the data. In this case, the data section contains TCP; in the above example, the bolded values are the regular portion of a TCP header, which is 20 bytes long without options. The TCP header contains various fields including the source and destination ports, sequence and acknowledgment numbers, window size, TCP flags, urgent pointer, and reserved bits.

Like the IP header, the TCP header may also contain options. And like the IP header, the TCP header’s length must be a multiple of 4 bytes. The final 8 bytes in the example, shown in plain text, are the TCP options. (Note that TCP options and IP options are two different things.) Because the TCP options change the length of the TCP header, the length is set in the header. In this case, the value of the first half of the 13th byte – in this case, 7 – means that the TCP header is made of seven 4-byte chunks, or a total of 28 bytes: 20 bytes of standard TCP header and 8 bytes of TCP options.

Because only one hex value is set aside for indicating the TCP header length, the maximum value that can be set is hex F, which is decimal 15. That means that fifteen 4-byte chunks, or 60 bytes, is the maximum length for the entire TCP header, including TCP options.

TCP Options

Let’s take a look at some commonly used TCP options:

Maximum Segment Size (MSS): The Maximum Segment Size (MSS) was defined in RFC 793; it is used to exchange Maximum Transfer Unit (MTU) / Maximum Receive Unit (MRU) sizes during the TCP three-way handshake. Basically, both ends of the connection will state the maximum IP datagram size that they can handle without using fragmentation, and the lower of the two values will be used by both ends. If the MSS option is omitted by one or both ends of the connection, the value 536 bytes will be used (per RFC 1122). The MSS is only negotiated during the SYN and SYN/ACK packets in the TCP three-way handshake, so the MSS option should only be seen in those packets.

Window Scale (WSCALE): Don’t confuse the TCP option called Window Scale with the standard TCP header field called Window Size. The window size is used to record how many bytes of buffer space the host has available for receiving data. Because the window size field is only 16 bits long, this limits the maximum window size to 65,535 bytes. At one time, this was sufficient, but it no longer is. In RFC 1072, the Window Scale (WSCALE) option was defined; it was redefined in RFC 1323. The Window Scale value is used to shift the window size field’s value up to a maximum value of approximately a gigabyte. Like the MSS option, the WSCALE option should only appear in SYN and SYN/ACK packets during the handshake. However, if both ends of the connection do not use the WSCALE option, the window sizes will remain unchanged.

Timestamp: The Timestamp option was first defined in its current form in RFC 1323. Its purpose is to track the round-trip delivery time for data in order to identify changes in latency that may require acknowledgment timer adjustments. Timestamps can only be used if both ends of the connection agree to use them, as you’ll see shortly. Unlike the MSS and WSCALE options, Timestamp options are typically used throughout a TCP session, so if they are being used, you should see them in most of the packets.

The Timestamp option has two timestamp fields: the Timestamp Value and the Timestamp Echo Reply. When a host first timestamps a packet in a connection, it puts the timestamp in the Timestamp Value field and leaves the Timestamp Echo Reply field set to 0 (generally). When the second host receives that packet and prepares to acknowledge it, it transfers the timestamp from the old packet’s Timestamp Value field to the new packet’s Timestamp Echo Reply field, and it puts a new timestamp in the Timestamp Value field. So the Timestamp Value field always contains the latest timestamp, while the Timestamp Echo Reply field contains the previous timestamp.

No Operation (NOP): The No Operation (NOP) TCP option, defined in RFC 793, is used to provide padding around other options. The length of the TCP header must be a multiple of 4 bytes; however, most of the options are not 4 bytes long. If the total length of the options is not a 4-byte multiple, one or more 1-byte NOPs will be added to the options in order to adjust the overall length. For example, if there were 6 bytes of options, two NOPs would be added. NOPs are sometimes used between options, particularly if an option needs to start on a certain byte boundary, so it is not unusual to see several NOPs throughout a set of TCP options.

Selective Acknowledgments: The concept of selective acknowledgments was defined in RFC 1072 and redefined in RFC 2018. Normally, when a host acknowledges data, it can only acknowledge the packets up to and including the sequence number immediately before a missing packet. This means that if a thousand packets are received but the second one is missing, the host can only acknowledge the receipt of the first packet, so the sender would have to resend all packets from number 2 through 1000. By using selective acknowledgments, the receiver could acknowledge the receipt of the packets from 3 through 1000, so the sender would only have to resend packet 2.

There are two TCP options for selective acknowledgments:

Selective Acknowledgment Permitted (SackOK): This option simply says that selective acknowledgments are permitted for this connection. SackOK must be included in the TCP options in both the SYN and SYN/ACK packets during the TCP three-way handshake, or it cannot be used. SackOK should not appear in any other packets.

Selective Acknowledgment Data: This option contains the actual acknowledgment information for a selective acknowledgment. It lists one or more pairs of sequence numbers, which define ranges of packets that are being acknowledged.

Analyzing Examples of TCP Options

Now we’ll return to the question that I asked at the beginning of the article. The TCP options in the example, a SYN packet from a Windows 98 system, are . What does that mean, and is it normal or not?

Here are the number of bytes that each of the options described above uses in a packet:

  • Maximum Segment Size (MSS): 4 bytes
  • Window Scale (WSCALE): 3 bytes
  • Timestamp (TS): 10 bytes
  • No Operation (NOP): 1 byte
  • Selective Acknowledgment Permitted (SackOK): 2 bytes
  • Selective Acknowledgment Data: 10 bytes (plus 8 bytes for each additional pair of sequence numbers)

Putting together all the items we’ve discussed, the options appear to be normal:

  • specifies a Maximum Segment size of 1460 bytes; it uses 4 bytes of the options. It is appropriate to see MSS used in a SYN packet.
  • is filler to provide padding between the mss 1460 and sackOK options. Each NOP takes up 1 byte of the header.
  • specifies that selective acknowledgments are permitted. This takes up 2 bytes of the header, and it is also appropriate to use in a SYN packet.

Here’s another example from a SYN packet sent by an OpenBSD 2.7 box: . Notice that it starts out with the same four options as the previous example (mss 1460,nop,nop,sackOK) but then adds several more. Here is the meaning of each of the additional fields, which total 16 bytes:

  • provides 1 byte of filler.
  • says that the Window Scale option is supported by the operating system but that it does not wish to use it. This takes up 3 bytes.
  • provides 2 bytes of filler.
  • is an initial timestamp. The Timestamp Value field is set to 50473, which is the time that the packet was sent. Since this is the first packet, the Timestamp Echo Reply field is set to 0, as there is no previous timestamp to place there. The timestamp option takes 10 bytes.

Another example of a SYN packet comes from a RedHat Linux 7.0 system: . Compare this example to the previous one; you’ll notice that both list the same 4 options: mss, sackOK, timestamp and wscale. But they list them in a different order, and they use nop instructions quite differently. Yet both are examples of normal traffic. The RFCs do not state what order the TCP options should be placed in, so each operating system is free to arrange them in whatever order they please.

The final example comes from a FreeBSD 4.0 box: for a SYN packet, the only option that it supplied was . This is quite different from the earlier examples, but it is perfectly acceptable; it just does not use the same options by default as the other operating systems did.

Conclusion

We have examined the normal use of TCP options. Various operating systems not only use different TCP options by default, but they list the options in different sequences, and they also use NOP’s for padding in different ways. This is not abnormal behavior, as long as the length of the options totals a multiple of 4 bytes, and the options make sense. For example, options such as MSS and Window Scale shouldn’t be set more than once in a packet, nor should they appear in packets that don’t have the SYN flag set.

Because most operating systems use a unique default TCP option combination, TCP options can be used to do “passive fingerprinting” of the operating system of a host that sends you a SYN packet. Also, systems can be fingerprinted based on what options they return in response to a set of options sent from another system. For example, if I send a SYN packet with the SackOK option set, and your system sends a SYN/ACK response without the SackOK option set, I know that your operating system does not support selective acknowledgments. By sending a SYN packet with many different options set and examining the options listed in the SYN/ACK response, I may be able to identify the operating system of the other host.

For more information on TCP options, two great references are TCP/IP Illustrated, Volume 1, by W. Richard Stevens, and Internet Core Protocols, by Eric A. Hall. More information on passive fingerprinting in general is available in Know Your Enemy: Passive Fingerprinting by Lance Spitzner, and information on fingerprinting using TCP options is available from Remote OS Detection Via TCP/IP Stack Fingerprinting by Fyodor.

Much thanks to Toby Miller for his suggestion to examine default TCP options.

Karen Kent Frederick is a senior security engineer for the Rapid Response Team at NFR Security. Karen has a B.S. in Computer Science and is currently completing her Master’s thesis in intrusion detection through the University of Idaho. She holds several certifications, including Microsoft Certified Systems Engineer + Internet, Check Point Certified Security Administrator, SANS GIAC Certified Intrusion Analyst, GIAC Certified Unix Security Administrator, and GIAC Certified Incident Handler. Karen is one of the authors and editors of “Intrusion Signatures and Analysis”, a new book on intrusion detection that was published in January 2001.

Relevant Links

Studying Normal Traffic, Part One
Karen Kent Frederick, SecurityFocus

Studying Normal Traffic, Part Two: Studying FTP Traffic
Karen Kent Frederick, SecurityFocus

Abnormal Traffic
Karen Kent Frederick, SecurityFocus

tcpdump
Tcpdump Group

WinDump
NetGroup

一个PXE装遍所有F5 v10版本

Posted on by
Reply

随着F5 V10版本的发布,安装方式出现了较大的变化,突然间没了像V9那样PXE安装,很多人都觉得不方便了. 随时总是携带一个USB DVDroom似乎不大现实,为每个版本做一个U盘代价也有点高,量产又不是总能成功. 幸运的是,尽管F5官方不再支持PXE安装,但是F5并没有将我们完全拒绝在PXE之外,本文教你如何通过一个虚拟系统同时实现V10所有版本的安装.

一、准备工作

在vmware等虚拟机中安装一个linux系统，例如centos，从网上下一个安装的ISO，带该5-6G左右，我安装的是centos 5，桥接接口到物理网卡。

系统要需安装dhcpd 服务，tftp服务，httpd服务，例如

dhcp-3.0.5-21.el5

httpd-2.2.3-31.el5.centos

tftp-server-0.49-2.el5.centos （ubuntu系统的默认tftp不行，改用dhcp-hpa，centos的就默认光盘带的即可）

dhcp服务提供F5启动时候获取IP用，tftp用于下载启动镜像，http服务用于实际安装时候的安装源地址

http、tftp服务保持启动后在后台运行，dhcp配好即可，后续的脚步会再用的时候主动去启动，免得干扰网络。

二、配置DHCP, tftp,http 等服务

/etc/dhcp.dconf配置：

ddns-update-style interim;

ignore client-updates;

subnet 192.168.1.0 netmask 255.255.255.0 {

# — default gateway

#        option routers                  192.168.1.1;

option subnet-mask              255.255.255.0;

option nis-domain               “domain.org”;

option domain-name              “domain.org”;

#        option domain-name-servers      192.168.1.1;

option time-offset              -18000; # Eastern Standard Time

#       option ntp-servers              192.168.1.1;

#       option netbios-name-servers     192.168.1.1;

# — Selects point-to-point node (default is hybrid). Don’t change this unless

# — you understand Netbios very well

#       option netbios-node-type 2;

range 192.168.1.2 192.168.1.3;

next-server 192.168.1.1;

filename “pxelinux.0″;

default-lease-time 21600;

max-lease-time 43200;

# You can ignore below fix address release setting.

host ltm3400-1 {

next-server 192.168.1.1;

hardware ethernet 00:24:e8:ed:46:66;

fixed-address 192.168.1.2;

filename “pxelinux.0″;

}

host ltm3400-2 {

next-server 192.168.1.1;

hardware ethernet 00:0c:29:78:45:af;

fixed-address 192.168.1.3;

filename “pxelinux.0″;

}

}

另外，需要给这个虚拟机的网卡配置ip为192.168.1.1/24，手工启动dhcp看有无配置错误

/etc/init.d/dhcpd start

如果启动不起来可以用命令检查：

[root@localhost f5lab]# /etc/init.d/dhcpd configtest

Syntax: OK

/etc/xinetd.d/tftp 配置

# default: off

# description: The tftp server serves files using the trivial file transfer \

#       protocol.  The tftp protocol is often used to boot diskless \

#       workstations, download configuration files to network-aware printers, \

#       and to start the installation process for some operating systems.

service tftp

{

socket_type             = dgram

protocol                = udp

wait                    = yes

user                    = root

server                  = /usr/sbin/in.tftpd

server_args             = -s /tftpboot -c

disable                 = no

per_source              = 11

cps                     = 100 2

flags                   = IPv4

}

确保tftp服务已启动:netstat –na | grep –i 69 ,如果没启动的话，用xinetd启动它：/etc/init.d/xinetd restart

Httpd服务没什么可配，启动即可。我们主要是借用一下http协议以便下载文件而已。但需要在网站根目录建立一个文件夹以便mount用，我的系统网站根目录是

[root@localhost xinetd.d]# cd /var/www/html/

所以在其下建立一个文件夹,名字叫 pxe10

mkdir pxe10

然后在/mnt 目录下也创建一个目录，名字叫iso10

mkdir /mnt/iso10

最后在home目录下创建一个目录f5lab，然后将v10.0.1, v10.1.0, v10.2.0等ISO文件拷贝到这个目录下

[root@localhost f5lab]# pwd

/home/f5lab

[root@localhost f5lab]# ll

total 2508712

-rw-r–r– 1 f5lab f5lab 523837440 Dec 11 03:44 BIGIP-10.0.1.283.0.iso

-rw-r–r– 1 f5lab f5lab 780675072 Dec 11 03:47 BIGIP-10.1.0.3341.0.iso

-rw-r–r– 1 f5lab f5lab 819111936 Dec 11 03:47 bigip_10.2.iso

-rw-r–r– 1 f5lab f5lab 142143488 Dec 11 03:47 Hotfix-BIGIP-10.1.0-3372.0-HF1.iso

-rw-r–r– 1 f5lab f5lab 146835456 Dec 11 03:46 Hotfix-BIGIP-10.2.0-1755.1-HF1.iso

-rw-r–r– 1 f5lab f5lab 153767936 Dec 11 03:46 Hotfix-BIGIP-10.2.0-1789.0-HF2.iso

三、写一个脚本

(脚本下载)pxeinstall

将这个文件放到/usr/bin下，设置权限

cd /usr/bin

chmod 755 pxeinstall.sh

(脚本里的文件名需要改为你的实际F5安装ISO文件名)

四、正式使用

正式用的时候，启动虚拟机，确保网卡已桥接，然后运行 pxeinstall.sh 脚本

[root@localhost f5lab]# pxeinstall.sh

Please input which version will be installed.

0 for 10.0.1 ; 1 for 10.1.0 ; 2 for 10.2.0 ; 3 for 10.2.1(Not ready):

脚本提示选择要安装的版本，例如选2 安装10.2,得到如下提示：

==========================================

You are installing 10.2.0.This scirpt can run more than one time.

==========================================

>>>Will start dhcp server….

>>>DHCP server start successfully!

==========================================

Please use below install source when running image2disk command

http://192.168.1.1/pxe10/iso10/

Finished PXE setting!

==========================================

好了，连接F5管理接口到你笔记本的接口，然后加电F5，开始安装：

asm tip (updating)

Posted on by
Reply

1.配置login page 强制功能时，被强制的页面一定要在allowed url中明确定义，否则会出现奇怪的结果。
定义allowed url，这url就是希望被强制的页面
定义login url ，需正确的判断成功的条件
定义login page setting，设置被强制的url或设置超时时间

2.配置navigation parameters时，定义的参数如果是for any url path的，那么用到这个参数的所有url都要在allowed url中明确定义，否则会被阻拦。

3.XFF TRUST, 默认没开启，即ASM不以http header中的x-forwarded-for作为源地址。开启后，asm总是以最后一个x-forwarded-for值为准，即：
a。如果有多行x-forwarded-for，则取最后一行，最后一行如果为空值，则以ip包源地址为准
b.如果一个x-forwarded-for头中有多个值，以最后一个IP为准。
c。如果x-forwarded-for头的值ASM无法分析（例如不是以逗号分隔ip，或者值为非ip），则以ip包源地址为准
d.如果vs的http profile开启了x-forwarded-for将导致ASM取得是ip包源地址

4.

  learn Alarm Block
transparent √ √  
blocking √ √ √
staging blocking √ √ ×**
transparent √ ×* ×
tightening blocking √ √ √**
transparent √ √* ×

*当策略为transparent，如果一个wildcard策略对象同时处在staging 和tightening阶段，此时asm的日志文件中不会记录该请求日志。

**当策略为blocking，如果一个wildcard策略对象同时处在staging和tightening阶段（v11.1测试无需同时处于该两个阶段，只需处于tightening阶段即可），此时促发violation的请求不会被block。例如parameter当前有一个处于tightening的wildcard对象，然后通过它新学到一个全新的parameter，此时新的parameter自动处于staging阶段，针对该新parameter ASM是不阻拦的。

logging profile ,控制是否记录request log到report–requesting里，不控制是否记录到asm.log。
blocking setting中的alarm控制是否写日志到asm.log中
blocking setting中的learn控制是否记录violation到 manual policy building中 （数据库PLC.LRN_REQUESTS）

5.

Tightening is using wildcards to learn the entities (file types, URLs,
parameters, and cookies). Staging is learning the attributes of an entity
(wildcard or explicit), providing additional granularity over tightening.

6.

F5 Networks recommends against using both tightening and
staging at the same time on the same wildcard entity.

7.选择production的方式建立一个fundamental type的策略，设置后的策略并检测到流量后开始进入自动学习模式：

（1）策略处于blocking模式，signature处于staging模式

（2）学习周期7天

（3）file type会自动增加一个wildcard 并置为tightening模式，可以自动将学习到的文件类型放入策略列表中（但也不是全部）

（4）url自动创建2个wildcard，但不处于学习状态，即不学习url

（5）parameters 自动创建wildcard，处于staging阶段，不能自动将参数放入策略列表中

（6）header-cookie,自动创建wildcard，不学习

7.1如果将fundamental type改为enhanced type 则parameters 进入tightening状态自动将学习到的条目加入策略列表，cookie也进入tightening状态。–似乎也不是所有条目都能自动放入到策略列表中。cookie学习到的还得等待手工accept。

7.2如果将enhaced type改为complete type 则又会增加URI进入tightening状态来自动将学习到的条目加入到列表。—-似乎也不是所有条目都能自动放入到策略列表中

（其实这些差别可以在自动策略学习设置界面中的“

Security Policy Element

部分看到”）

8.建立策略的时候 选择 production site 和QA lab 两种模式，对策略的学习到底有何不同的影响？

QA自动认为所有请求都来自合法用户，QA会将学习到的所有项目都列入策略作为合法条目，即staging-tightening summary界面的“have suggestions”会很少，但不是没有。

production缺省认为请求不是来自合法用户，需要手工设置trust ip，此模式下从非信任IP上学习到的项目部分能自动进入策略列表，但也有很多等待手工确认。 没太明白这里。

浅谈大型网站动态应用系统[zt]

Posted on by
Reply

动态应用，是相对于网站静态内容而言，是指以c/c++、php、Java、perl、.net等服务器端语言开发的网络应用软件，比如论坛、网络相册、交友、BLOG等常见应用。动态应用系统通常与数据库系统、缓存系统、分布式存储系统等密不可分。

大型动态应用系统平台主要是针对于大流量、高并发网站建立的底层系统架构。大型网站的运行需要一个可靠、安全、可扩展、易维护的应用系统平台做为支撑，以保证网站应用的平稳运行。

大型动态应用系统又可分为几个子系统：

1）Web前端系统

2）负载均衡系统

3）数据库集群系统

4）缓存系统

5）分布式存储系统

6）分布式服务器管理系统

7）代码分发系统

Web前端系统

结构图：

webe5898de7abafe7b3bbe7bb9f1

为了达到不同应用的服务器共享、避免单点故障、集中管理、统一配置等目的，不以应用划分服务器，而是将所有服务器做统一使用，每台服务器都可以对多个应用提供服务，当某些应用访问量升高时，通过增加服务器节点达到整个服务器集群的性能提高，同时使他应用也会受益。该Web前端系统基于Apache/Lighttpd/Eginx等的虚拟主机平台，提供PHP程序运行环境。服务器对开发人员是透明的，不需要开发人员介入服务器管理

负载均衡系统

结构图：

e8b49fe8bdbde59d87e8a1a1e7b3bbe7bb9f

负载均衡系统分为硬件和软件两种。硬件负载均衡效率高，但是价格贵，比如F5等。软件负载均衡系统价格较低或者免费，效率较硬件负载均衡系统低，不过对于流量一般或稍大些网站来讲也足够使用，比如lvs, nginx。大多数网站都是硬件、软件负载均衡系统并用。

数据库集群系统

结构图：

e695b0e68daee5ba93e99b86e7bea4e7b3bbe7bb9f

由于Web前端采用了负载均衡集群结构提高了服务的有效性和扩展性，因此数据库必须也是高可靠的，才能保证整个服务体系的高可靠性，如何构建一个高可靠的、可以提供大规模并发处理的数据库体系？

我们可以采用如上图所示的方案：

1) 使用 MySQL 数据库，考虑到Web应用的数据库读多写少的特点，我们主要对读数据库做了优化，提供专用的读数据库和写数据库，在应用程序中实现读操作和写操作分别访问不同的数据库。

2) 使用 MySQL Replication 机制实现快速将主库（写库）的数据库复制到从库（读库）。一个主库对应多个从库，主库数据实时同步到从库。

3) 写数据库有多台，每台都可以提供多个应用共同使用，这样可以解决写库的性能瓶颈问题和单点故障问题。

4) 读数据库有多台，通过负载均衡设备实现负载均衡，从而达到读数据库的高性能、高可靠和高可扩展性。

5) 数据库服务器和应用服务器分离。

6) 从数据库使用BigIP做负载均衡。

缓存系统

结构图：

e7bc93e5ad98e7b3bbe7bb9f

缓存分为文件缓存、内存缓存、数据库缓存。在大型Web应用中使用最多且效率最高的是内存缓存。最常用的内存缓存工具是Memcached。使用正确的缓存系统可以达到实现以下目标：

1）使用缓存系统可以提高访问效率，提高服务器吞吐能力，改善用户体验。

2）减轻对数据库及存储集服务器的访问压力。

3）Memcached服务器有多台，避免单点故障，提供高可靠性和可扩展性，提高性能。

分布式存储系统

结构图：

e58886e5b883e5bc8fe5ad98e582a8e7b3bbe7bb9f1

Web系统平台中的存储需求有下面两个特点：

1) 存储量很大，经常会达到单台服务器无法提供的规模，比如相册、视频等应用。因此需要专业的大规模存储系统。

2) 负载均衡cluster中的每个节点都有可能访问任何一个数据对象，每个节点对数据的处理也能被其他节点共享，因此这些节点要操作的数据从逻辑上看只能是一个整体，不是各自独立的数据资源。

因此高性能的分布式存储系统对于大型网站应用来说是非常重要的一环。（这个地方需要加入对某个分布式存储系统的简单介绍。）

分布式服务器管理系统

结构图：

e58886e5b883e5bc8fe69c8de58aa1e599a8e7aea1e79086e7b3bbe7bb9f 

随着网站访问流量的不断增加，大多的网络服务都是以负载均衡集群的方式对外提供服务，随之集群规模的扩大，原来基于单机的服务器管理模式已经不能够满足我们的需求，新的需求必须能够集中式的、分组的、批量的、自动化的对服务器进行管理，能够批量化的执行计划任务。

在分布式服务器管理系统软件中有一些比较优秀的软件，其中比较理想的一个是Cfengine。它可以对服务器进行分组，不同的分组可以分别定制系统配置文件、计划任务等配置。它是基于C/S 结构的，所有的服务器配置和管理脚本程序都保存在Cfengine Server上，而被管理的服务器运行着 Cfengine Client 程序，Cfengine Client通过SSL加密的连接定期的向服务器端发送请求以获取最新的配置文件和管理命令、脚本程序、补丁安装等任务。

有了Cfengine这种集中式的服务器管理工具，我们就可以高效的实现大规模的服务器集群管理，被管理服务器和 Cfengine Server 可以分布在任何位置，只要网络可以连通就能实现快速自动化的管理。

代码发布系统

结构图：

e4bba3e7a081e58f91e5b883e7b3bbe7bb9f_e589afe69cac

随着网站访问流量的不断增加，大多的网络服务都是以负载均衡集群的方式对外提供服务，随之集群规模的扩大，为了满足集群环境下程序代码的批量分发和更新，我们还需要一个程序代码发布系统。

这个发布系统可以帮我们实现下面的目标：

1) 生产环境的服务器以虚拟主机方式提供服务，不需要开发人员介入维护和直接操作，提供发布系统可以实现不需要登陆服务器就能把程序分发到目标服务器。

2) 我们要实现内部开发、内部测试、生产环境测试、生产环境发布的4个开发阶段的管理，发布系统可以介入各个阶段的代码发布。

3) 我们需要实现源代码管理和版本控制，SVN可以实现该需求。

这里面可以使用常用的工具Rsync，通过开发相应的脚本工具实现服务器集群间代码同步分发。

http://www.isn8.com/blog/?p=857

扫盲计划之：log&syslog

Posted on by
1

GUI下的log显示有：

system———->/var/log/messages

packer filter ————->/var/log/pktfilter

Local traffic————–>/var/log/ltm

audit—————–>/var/log/audit

打开GUI下的审计需要打开MCP审计功能并确保MCP的日志级别不高于notice

打开b 命令的审计则需打开bigpipe的审计

在/var/log 下其实还有很多其他日志文件

——————————————————

使用logtool可以快速的在所有日志中查找感兴趣的日志，使用方法如下

[root@v10-1:Active] log # logtool –help
Unknown option: help
Usage: /usr/bin/logtool [options] ["<SearchString>"]
Options:
–filename:    Append the log file name to the message
–level=LEVEL: Search for messages at a level
–system=NAME: Search for messages from a system
–slot=NUMBER: Search for messages from a slot
–zipped:      Search for messages in compressed log files

注意searchstring是区分大小写的。

—————————————-

在命令行下查看日志时候可以通过增加 bigcodes 管道来解析F5一些专用的抽象代码,例如

cat /var/log/ltm | bigcodes |less

—————————————-

resize-logFS 可以用来resize 给/var/log预分配的固定空间，默认是7G，可配区间是1-10G

——————–

配置syslog-ng将log发送到远程syslog服务器

Continue reading →