Monday, January 17, 2011

Route - Linux

Name

route - show / manipulate the IP routing table

Synopsis

route [-CFvnee]

route
[-v] [-A family] add [-net|-host] target [netmask Nm] [gw Gw] [metric N] [mss M] [window W] [irtt I] [reject] [mod] [dyn] [reinstate] [[dev] If]
route
[-v] [-A family] del [-net|-host] target [gw Gw] [netmask Nm] [metric N] [[dev] If]
route
[-V] [--version] [-h] [--help]

Description

Route manipulates the kernel's IP routing tables. Its primary use is to set up static routes to specific hosts or networks via an interface after it has been configured with the ifconfig(8) program.

When the add or del options are used, route modifies the routing tables. Without these options, route displays the current contents of the routing tables.

Options

-A family
use the specified address family (eg 'inet'; use 'route --help' for a full list).
-F
operate on the kernel's FIB (Forwarding Information Base) routing table. This is the default.
-C
operate on the kernel's routing cache.
-v
select verbose operation.
-n
show numerical addresses instead of trying to determine symbolic host names. This is useful if you are trying to determine why the route to your nameserver has vanished.
-e
use netstat(8)-format for displaying the routing table. -ee will generate a very long line with all parameters from the routing table.
del
delete a route.
add
add a new route.
target
the destination network or host. You can provide IP addresses in dotted decimal or host/network names.
-net
the target is a network.
-host
the target is a host.
netmask NM
when adding a network route, the netmask to be used.
gw GW
route packets via a gateway. NOTE: The specified gateway must be reachable first. This usually means that you have to set up a static route to the gateway beforehand. If you specify the address of one of your local interfaces, it will be used to decide about the interface to which the packets should be routed to. This is a BSDism compatibility hack.
metric M
set the metric field in the routing table (used by routing daemons) to M.
mss M
set the TCP Maximum Segment Size (MSS) for connections over this route to M bytes. The default is the device MTU minus headers, or a lower MTU when path mtu discovery occured. This setting can be used to force smaller TCP packets on the other end when path mtu discovery does not work (usually because of misconfigured firewalls that block ICMP Fragmentation Needed)
window W
set the TCP window size for connections over this route to W bytes. This is typically only used on AX.25 networks and with drivers unable to handle back to back frames.
irtt I
set the initial round trip time (irtt) for TCP connections over this route to I milliseconds (1-12000). This is typically only used on AX.25 networks. If omitted the RFC 1122 default of 300ms is used.
reject
install a blocking route, which will force a route lookup to fail. This is for example used to mask out networks before using the default route. This is NOT for firewalling.
mod, dyn, reinstate
install a dynamic or modified route. These flags are for diagnostic purposes, and are generally only set by routing daemons.
dev If
force the route to be associated with the specified device, as the kernel will otherwise try to determine the device on its own (by checking already existing routes and device specifications, and where the route is added to). In most normal networks you won't need this.

If dev If is the last option on the command line, the word dev may be omitted, as it's the default. Otherwise the order of the route modifiers (metric - netmask - gw - dev) doesn't matter.

Examples

route add -net 127.0.0.0
adds the normal loopback entry, using netmask 255.0.0.0 (class A net, determined from the destination address) and associated with the "lo" device (assuming this device was prviously set up correctly with ifconfig(8)).
route add -net 192.56.76.0 netmask 255.255.255.0 dev eth0
adds a route to the network 192.56.76.x via "eth0". The Class C netmask modifier is not really necessary here because 192.* is a Class C IP address. The word "dev" can be omitted here.
route add default gw mango-gw
adds a default route (which will be used if no other route matches). All packets using this route will be gatewayed through "mango-gw". The device which will actually be used for that route depends on how we can reach "mango-gw" - the static route to "mango-gw" will have to be set up before.
route add ipx4 sl0
Adds the route to the "ipx4" host via the SLIP interface (assuming that "ipx4" is the SLIP host).
route add -net 192.57.66.0 netmask 255.255.255.0 gw ipx4
This command adds the net "192.57.66.x" to be gatewayed through the former route to the SLIP interface.
route add -net 224.0.0.0 netmask 240.0.0.0 dev eth0
This is an obscure one documented so people know how to do it. This sets all of the class D (multicast) IP routes to go via "eth0". This is the correct normal configuration line with a multicasting kernel.
route add -net 10.0.0.0 netmask 255.0.0.0 reject
This installs a rejecting route for the private network "10.x.x.x."

Output

The output of the kernel routing table is organized in the following columns
Destination
The destination network or destination host.
Gateway
The gateway address or '*' if none set.
Genmask
The netmask for the destination net; '255.255.255.255' for a host destination and '0.0.0.0' for the default route.
Flags
Possible flags include
U (route is up)
H (target is a host)
G (use gateway)
R (reinstate route for dynamic routing)
D (dynamically installed by daemon or redirect)
M (modified from routing daemon or redirect)
A (installed by addrconf)
C (cache entry)
! (reject route)
Metric
The 'distance' to the target (usually counted in hops). It is not used by recent kernels, but may be needed by routing daemons.
Ref
Number of references to this route. (Not used in the Linux kernel.)
Use
Count of lookups for the route. Depending on the use of -F and -C this will be either route cache misses (-F) or hits (-C).
Iface
Interface to which packets for this route will be sent.
MSS
Default maximum segement size for TCP connections over this route.
Window
Default window size for TCP connections over this route.
irtt
Initial RTT (Round Trip Time). The kernel uses this to guess about the best TCP protocol parameters without waiting on (possibly slow) answers.
HH (cached only)
The number of ARP entries and cached routes that refer to the hardware header cache for the cached route. This will be -1 if a hardware address is not needed for the interface of the cached route (e.g. lo).
Arp (cached only)
Whether or not the hardware address for the cached route is up to date.

Files

/proc/net/ipv6_route
/proc/net/route
/proc/net/rt_cache

See Also

Saturday, January 15, 2011

Curl - Command Line Web Browser Client

url is a tool to transfer data from or to a server, using one of the supported protocols (HTTP, HTTPS, FTP, FTPS, TFTP, DICT, TELNET, LDAP or FILE). The command is designed to work without user interaction.

Syntax

curl [options] [URL...]

-a/--append(FTP) When used in an FTP upload, this will tell curl to append to the target file instead of overwriting it. If the file doesn't exist, it will be created.

If this option is used twice, the second one will disable append mode again.

-A/--user-agent (HTTP) Specify the User-Agent string to send to the HTTP server. Some badly done CGIs fail if its not set to "Mozilla/4.0". To encode blanks in the string, surround the string with single quote marks. This can also be set with the -H/--header option of course.

If this option is set more than once, the last one will be the one that's used.

--anyauth(HTTP) Tells curl to figure out authentication method by itself, and use the most secure one the remote site claims it supports. This is done by first doing a request and checking the response-headers, thus inducing an extra network round-trip. This is used instead of setting a specific authentication method, which you can do with --basic, --digest, --ntlm, and --negotiate.

Note that using --anyauth is not recommended if you do uploads from stdin, since it may require data to be sent twice and then the client must be able to rewind. If the need should arise when uploading from stdin, the upload operation will fail.

If this option is used several times, the following occurrences make no difference.

-b/--cookie (HTTP) Pass the data to the HTTP server as a cookie. It is supposedly the data previously received from the server in a "Set-Cookie:" line. The data should be in the format "NAME1=VALUE1; NAME2=VALUE2".

If no '=' letter is used in the line, it is treated as a filename to use to read previously stored cookie lines from, which should be used in this session if they match. Using this method also activates the "cookie parser" which will make curl record incoming cookies too, which may be handy if you're using this in combination with the -L/--location option. The file format of the file to read cookies from should be plain HTTP headers or the Netscape/Mozilla cookie file format.

NOTE that the file specified with -b/--cookie is only used as input. No cookies will be stored in the file. To store cookies, use the -c/--cookie-jar option or you could even save the HTTP headers to a file using -D/--dump-header!

If this option is set more than once, the last one will be the one that's used.

-B/--use-asciiEnable ASCII transfer when using FTP or LDAP. For FTP, this can also be enforced by using an URL that ends with ";type=A". This option causes data sent to stdout to be in text mode for win32 systems.

If this option is used twice, the second one will disable ASCII usage.

--basic(HTTP) Tells curl to use HTTP Basic authentication. This is the default and this option is usually pointless, unless you use it to override a previously set option that sets a different authentication method (such as --ntlm, --digest and --negotiate).

If this option is used several times, the following occurrences make no difference.

--ciphers (SSL) Specifies which ciphers to use in the connection. The list of ciphers must be using valid ciphers. Read up on SSL cipher list details on this URL: http://www.openssl.org/docs/apps/ciphers.html

If this option is used several times, the last one will override the others.

--compressed(HTTP) Request a compressed response using one of the algorithms libcurl supports, and return the uncompressed document. If this option is used and the server sends an unsupported encoding, Curl will report an error.

If this option is used several times, each occurrence will toggle it on/off.
--connect-timeout Maximum time in seconds that you allow the connection to the server to take. This only limits the connection phase, once curl has connected this option is of no more use. See also the -m/--max-time option.

If this option is used several times, the last one will be used.

-c/--cookie-jar Specify to which file you want curl to write all cookies after a completed operation. Curl writes all cookies previously read from a specified file as well as all cookies received from remote server(s). If no cookies are known, no file will be written. The file will be written using the Netscape cookie file format. If you set the file name to a single dash, "-", the cookies will be written to stdout.

NOTE If the cookie jar can't be created or written to, the whole curl operation won't fail or even report an error clearly. Using -v will get a warning displayed, but that is the only visible feedback you get about this possibly lethal situation.

If this option is used several times, the last specified file name will be used.

-C/--continue-at Continue/Resume a previous file transfer at the given offset. The given offset is the exact number of bytes that will be skipped counted from the beginning of the source file before it is transferred to the destination. If used with uploads, the ftp server command SIZE will not be used by curl.

Use "-C -" to tell curl to automatically find out where/how to resume the transfer. It then uses the given output/input files to figure that out.

If this option is used several times, the last one will be used.

--create-dirsWhen used in conjunction with the -o option, curl will create the necessary local directory hierarchy as needed. This option creates the dirs mentioned with the -o option, nothing else. If the -o file name uses no dir or if the dirs it mentions already exist, no dir will be created.

To create remote directories when using FTP, try --ftp-create-dirs.

--crlf (FTP)Convert LF to CRLF in upload. Useful for MVS (OS/390).

If this option is used several times, the following occurrences make no difference.

-d/--data (HTTP) Sends the specified data in a POST request to the HTTP server, in a way that can emulate as if a user has filled in a HTML form and pressed the submit button. Note that the data is sent exactly as specified with no extra processing (with all newlines cut off). The data is expected to be "url-encoded". This will cause curl to pass the data to the server using the content-type application/x-www-form-urlencoded. Compare to -F/--form. If this option is used more than once on the same command line, the data pieces specified will be merged together with a separating &-letter. Thus, using '-d name=daniel -d skill=lousy' would generate a post chunk that looks like 'name=daniel&skill=lousy'.

If you start the data with the letter @, the rest should be a file name to read the data from, or - if you want curl to read the data from stdin. The contents of the file must already be url-encoded. Multiple files can also be specified. Posting data from a file named 'foobar' would thus be done with --data @foo-bar".

To post data purely binary, you should instead use the --data-binary option.

-d/--data is the same as --data-ascii.

If this option is used several times, the ones following the first will append data.

--data-ascii (HTTP) This is an alias for the -d/--data option.

If this option is used several times, the ones following the first will append data.

--data-binary (HTTP) This posts data in a similar manner as --data-ascii does, although when using this option the entire context of the posted data is kept as-is. If you want to post a binary file without the strip-newlines feature of the --data-ascii option, this is for you.

If this option is used several times, the ones following the first will append data.

--digest(HTTP) Enables HTTP Digest authentication. This is a authentication that prevents the password from being sent over the wire in clear text. Use this in combination with the normal -u/--user option to set user name and password. See also --ntlm, --negoti- ate and --anyauth for related options.

If this option is used several times, the following occurrences make no difference.

--disable-eprt(FTP) Tell curl to disable the use of the EPRT and LPRT commands when doing active FTP transfers. Curl will normally always first attempt to use EPRT, then LPRT before using PORT, but with this option, it will use PORT right away. EPRT and LPRT are extensions to the original FTP protocol, may not work on all servers but enable more functionality in a better way than the traditional PORT command.

If this option is used several times, each occurrence will toggle this on/off.

--disable-epsv(FTP) Tell curl to disable the use of the EPSV command when doing passive FTP transfers. Curl will normally always first attempt to use EPSV before PASV, but with this option, it will not try using EPSV.

If this option is used several times, each occurrence will toggle this on/off.

-D/--dump-header Write the protocol headers to the specified file.

This option is handy to use when you want to store the headers that a HTTP site sends to you. Cookies from the headers could then be read in a second curl invoke by using the -b/--cookie option! The -c/--cookie-jar option is however a better way to store cookies.

When used on FTP, the ftp server response lines are considered being "headers" and thus are saved there.

If this option is used several times, the last one will be used.

-e/--referer (HTTP) Sends the "Referer Page" information to the HTTP server. This can also be set with the -H/--header flag of course. When used with -L/--location you can append ";auto" to the --referer URL to make curl automatically set the previous URL when it follows a Location: header. The ";auto" string can be used alone, even if you don't set an initial --referer.

If this option is used several times, the last one will be used.

--engine Select the OpenSSL crypto engine to use for cipher operations. Use --engine list to print a list of build-time supported engines. Note that not all (or none) of the engines may be available at run-time.
--environment(RISC OS ONLY) Sets a range of environment variables, using the names the -w option supports, to easier allow extraction of useful information after having run curl.

If this option is used several times, each occurrence will toggle this on/off.

--egd-file (HTTPS) Specify the path name to the Entropy Gathering Daemon socket. The socket is used to seed the random engine for SSL connections. See also the --random-file option.
-E/--cert (HTTPS) Tells curl to use the specified certificate file when getting a file with HTTPS. The certificate must be in PEM format. If the optional password isn't specified, it will be queried for on the terminal. Note that this certificate is the private key and the private certificate concatenated!

If this option is used several times, the last one will be used.

--cert-type (SSL) Tells curl what certificate type the provided certificate is in. PEM, DER and ENG are recognized types.

If this option is used several times, the last one will be used.

--cacert (HTTPS) Tells curl to use the specified certificate file to verify the peer. The file may contain multiple CA certificates. The certificate(s) must be in PEM format.

curl recognizes the environment variable named 'CURL_CA_BUNDLE' if that is set, and uses the given path as a path to a CA cert bundle. This option overrides that variable.

The windows version of curl will automatically look for a CA certs file named 'curl-ca-bundle.crt', either in the same directory as curl.exe, or in the Current Working Directory, or in any folder along your PATH.

If this option is used several times, the last one will be used.

--capath (HTTPS) Tells curl to use the specified certificate directory to verify the peer. The certificates must be in PEM format, and the directory must have been processed using the c_rehash utility supplied with openssl. Using --capath can allow curl to make https connections much more efficiently than using --cacert if the --cacert file contains many CA certificates.

If this option is used several times, the last one will be used.

-f/--fail(HTTP) Fail silently (no output at all) on server errors. This is mostly done like this to better enable scripts etc to better deal with failed attempts. In normal cases when a HTTP server fails to deliver a document, it returns an HTML document stating so (which often also describes why and more). This flag will prevent curl from outputting that and return error 22.

If this option is used twice, the second will again disable silent failure.

--ftp-account [data](FTP) When an FTP server asks for "account data" after user name and password has been provided, this data is sent off using the ACCT command. (Added in 7.13.0)

If this option is used twice, the second will override the previous use.

--ftp-create-dirs(FTP) When an FTP URL/operation uses a path that doesn't currently exist on the server, the standard behavior of curl is to fail. Using this option, curl will instead attempt to create missing directories.

If this option is used twice, the second will again disable directory creation.

--ftp-method [method](FTP) Control what method curl should use to reach a file on a FTP(S) server. The method argument should be one of the following alternatives:

multicwd
curl does a single CWD operation for each path part in the given URL. For deep hierarchies this means very many commands. This is how RFC1738 says it should be done. This is the default but the slowest behavior.

nocwd curl does no CWD at all. curl will do SIZE, RETR, STOR
etc and give a full path to the server for all these commands. This is the fastest behavior.

singlecwd
curl does one CWD with the full target directory and then operates on the file "normally" (like in the multicwd case). This is somewhat more standards compliant than 'nocwd' but without the full penalty of 'multicwd'.

--ftp-pasv(FTP) Use PASV when transferring. PASV is the internal default behavior, but using this option can be used to override a previous --ftp-port option. (Added in 7.11.0)

If this option is used several times, the following occurrences make no difference.

--ftp-alternative-to-user (FTP) If authenticating with the USER and PASS commands fails, send this command. When connecting to Tumbleweed's Secure Transport server over FTPS using a client certificate, using "SITE AUTH" will tell the server to retrieve the username from the certificate. (Added in 7.15.5)
--ftp-skip-pasv-ip(FTP) Tell curl to not use the IP address the server suggests in its response to curl's PASV command when curl connects the data connection. Instead curl will re-use the same IP address it already uses for the control connection. (Added in 7.14.2)

This option has no effect if PORT, EPRT or EPSV is used instead of PASV.

If this option is used twice, the second will again use the server's suggested address.

--ftp-ssl(FTP) Try to use SSL/TLS for the FTP connection. Reverts to a non-secure connection if the server doesn't support SSL/TLS. (Added in 7.11.0)

If this option is used twice, the second will again disable this.

--ftp-ssl-reqd(FTP) Require SSL/TLS for the FTP connection. Terminates the connection if the server doesn't support SSL/TLS. (Added in 7.15.5)

If this option is used twice, the second will again disable this.

-F/--form (HTTP) This lets curl emulate a filled in form in which a user has pressed the submit button. This causes curl to POST data using the Content-Type multipart/form-data according to RFC1867. This enables uploading of binary files etc. To force the 'content' part to be a file, prefix the file name with an @ sign. To just get the content part from a file, prefix the file name with the letter <. The difference between @ and <>Example, to send your password file to the server, where 'password' is the name of the form-field to which /etc/passwd will be the input:

curl -F password=@/etc/passwd www.mypasswords.com

To read the file's content from stdin instead of a file, use - where the file name should've been. This goes for both @ and <>

You can also tell curl what Content-Type to use by using 'type=', in a manner similar to:

curl -F "web=@index.html;type=text/html" url.com

or

curl -F "name=daniel;type=text/foo" url.com

You can also explicitly change the name field of an file upload part by setting filename=, like this:

curl -F "file=@localfile;filename=nameinpost" url.com

See further examples and details in the MANUAL.

This option can be used multiple times.

--form-string (HTTP) Similar to --form except that the value string for the named parameter is used literally. Leading '@' and '<' characters, and the ';type=' string in the value have no special meaning. Use this in preference to --form if there's any possibility that the string value may accidentally trigger the '@' or '<' features of --form.
-g/--globoffThis option switches off the "URL globbing parser". When you set this option, you can specify URLs that contain the letters {}[] without having them being interpreted by curl itself. Note that these letters are not normal legal URL contents but they should be encoded according to the URI standard.
-G/--getWhen used, this option will make all data specified with -d/--data or --data-binary to be used in a HTTP GET request instead of the POST request that otherwise would be used. The data will be appended to the URL with a '?' separator.

If used in combination with -I, the POST data will instead be appended to the URL with a HEAD request.

If this option is used several times, the following occurrences make no difference.

-H/--header
(HTTP) Extra header to use when getting a web page. You may specify any number of extra headers. Note that if you should add a custom header that has the same name as one of the internal ones curl would use, your externally set header will be used instead of the internal one. This allows you to make even trickier stuff than curl would normally do. You should not replace internally set headers without knowing perfectly well what you're doing. Replacing an internal header with one without content on the right side of the colon will prevent that header from appearing.

curl will make sure that each header you add/replace get sent with the proper end of line marker, you should thus not add that as a part of the header content: do not add newlines or carriage returns they will only mess things up for you.

See also the -A/--user-agent and -e/--referer options.

This option can be used multiple times to add/replace/remove multiple headers.

--ignore-content-length(HTTP) Ignore the Content-Length header. This is particularly useful for servers running Apache 1.x, which will report incorrect Content-Length for files larger than 2 gigabytes.
-i/--include(HTTP) Include the HTTP-header in the output. The HTTP-header includes things like server-name, date of the document, HTTP- version and more...

If this option is used twice, the second will again disable header include.

--interface Perform an operation using a specified interface. You can enter interface name, IP address or host name. An example could look like:

curl --interface eth0:1 http://www.netscape.com/

If this option is used several times, the last one will be used.

-I/--head(HTTP/FTP/FILE) Fetch the HTTP-header only! HTTP-servers feature the command HEAD which this uses to get nothing but the header of a document. When used on a FTP or FILE file, curl displays the file size and last modification time only.

If this option is used twice, the second will again disable header only.

-j/--junk-session-cookies(HTTP) When curl is told to read cookies from a given file, this option will make it discard all "session cookies". This will basically have the same effect as if a new session is started. Typical browsers always discard session cookies when they're closed down.

If this option is used several times, each occurrence will toggle this on/off.

-k/--insecure(SSL) This option explicitly allows curl to perform "insecure" SSL connections and transfers. All SSL connections are attempted to be made secure by using the CA certificate bundle installed by default. This makes all connections considered "insecure" to fail unless -k/--insecure is used.

If this option is used twice, the second time will again disable it.

--key (SSL) Private key file name. Allows you to provide your private key in this separate file.

If this option is used several times, the last one will be used.

--key-type (SSL) Private key file type. Specify which type your --key provided private key is. DER, PEM and ENG are supported.

If this option is used several times, the last one will be used.

--krb4 (FTP) Enable kerberos4 authentication and use. The level must be entered and should be one of 'clear', 'safe', 'confidential' or 'private'. Should you use a level that is not one of these, 'private' will instead be used.

This option requires that the library was built with kerberos4 support. This is not very common. Use -V/--version to see if your curl supports it.

If this option is used several times, the last one will be used.

-K/--config Specify which config file to read curl arguments from. The config file is a text file in which command line arguments can be written which then will be used as if they were written on the actual command line. Options and their parameters must be specified on the same config file line. If the parameter is to contain white spaces, the parameter must be enclosed within quotes. If the first column of a config line is a '#' character, the rest of the line will be treated as a comment.

Specify the filename as '-' to make curl read the file from stdin.

Note that to be able to specify a URL in the config file, you need to specify it using the --url option, and not by simply writing the URL on its own line. So, it could look similar to

this:

url = "http://curl.haxx.se/docs/"

This option can be used multiple times.

When curl is invoked, it always (unless -q is used) checks for a default config file and uses it if found. The default config file is checked for in the following places in this order:

1) curl tries to find the "home dir": It first checks for the CURL_HOME and then the HOME environment variables. Failing that, it uses getpwuid() on unix-like systems (which returns the home dir given the current user in your system). On Windows, it then checks for the APPDATA variable, or as a last resort the '%USER-PROFILE%0lication Data'.

2) On windows, if there is no _curlrc file in the home dir, it checks for one in the same dir the executable curl is placed. On unix-like systems, it will simply try to load .curlrc from the determined home dir.

--limit-rate Specify the maximum transfer rate you want curl to use. This feature is useful if you have a limited pipe and you'd like your transfer not use your entire bandwidth.

The given speed is measured in bytes/second, unless a suffix is appended. Appending 'k' or 'K' will count the number as kilobytes, 'm' or M' makes it megabytes while 'g' or 'G' makes it gigabytes. Examples: 200K, 3m and 1G.

If you are also using the -Y/--speed-limit option, that option will take precedence and might cripple the rate-limiting slightly, to help keeping the speed-limit logic working.

If this option is used several times, the last one will be used.

-l/--list-only(FTP) When listing an FTP directory, this switch forces a name-
only view. Especially useful if you want to machine-parse the
contents of an FTP directory since the normal directory view
doesn't use a standard look or format.

This option causes an FTP NLST command to be sent. Some FTP
servers list only files in their response to NLST; they do not
include subdirectories and symbolic links.

If this option is used twice, the second will again disable list
only.
--local-port [-num]Set a preferred number or range of local port numbers to use for the connection(s). Note that port numbers by nature is a scarce resource that will be busy at times so setting this range to something too narrow might cause unnecessary connection setup failures. (Added in 7.15.2)
-L/--location(HTTP/HTTPS) If the server reports that the requested page has moved to a different location (indicated with a Location: header and a 3XX response code) this option will make curl redo the request on the new place. If used together with -i/--include or -I/--head, headers from all requested pages will be shown. When authentication is used, curl only sends its credentials to the initial host. If a redirect takes curl to a different host, it won't be able to intercept the user+password. See also --location-trusted on how to change this. You can limit the amount of redirects to follow by using the --max-redirs option.

If this option is used twice, the second will again disable location following.

--location-trusted(HTTP/HTTPS) Like -L/--location, but will allow sending the name + password to all hosts that the site may redirect to. This may or may not introduce a security breach if the site redirects you do a site to which you'll send your authentication info (which is plaintext in the case of HTTP Basic authentication).

If this option is used twice, the second will again disable location following.

--max-filesize Specify the maximum size (in bytes) of a file to download. If the file requested is larger than this value, the transfer will not start and curl will return with exit code 63.

NOTE: The file size is not always known prior to download, and for such files this option has no effect even if the file transfer ends up being larger than this given limit. This concerns both FTP and HTTP transfers.

-m/--max-time Maximum time in seconds that you allow the whole operation to take. This is useful for preventing your batch jobs from hanging for hours due to slow networks or links going down. See also the --connect-timeout option.

If this option is used several times, the last one will be used.

-n/--netrcMakes curl scan the .netrc file in the user's home directory for login name and password. This is typically used for ftp on unix. If used with http, curl will enable user authentication. See netrc(4) or ftp for details on the file format. Curl will not complain if that file hasn't the right permissions (it should not be world nor group readable). The environment variable "HOME" is used to find the home directory.

A quick and very simple example of how to setup a .netrc to allow curl to ftp to the machine host.domain.com with user name 'myself' and password 'secret' should look similar to:

machine host.domain.com login myself password secret

If this option is used twice, the second will again disable netrc usage.

--negotiate(HTTP) Enables GSS-Negotiate authentication. The GSS-Negotiate method was designed by Microsoft and is used in their web applications. It is primarily meant as a support for Kerberos5 authentication but may be also used along with another authentication methods. For more information see IETF draft draft-brezak-spnego-http-04.txt.

This option requires that the library was built with GSSAPI support. This is not very common. Use -V/--version to see if your version supports GSS-Negotiate.

When using this option, you must also provide a fake -u/--user option to activate the authentication code properly. Sending a '-u :' is enough as the user name and password from the -u option aren't actually used.

If this option is used several times, the following occurrences make no difference.

-N/--no-bufferDisables the buffering of the output stream. In normal work situations, curl will use a standard buffered output stream that will have the effect that it will output the data in chunks, not necessarily exactly when the data arrives. Using this option will disable that buffering.

If this option is used twice, the second will again switch on buffering.

--ntlm (HTTP)Enables NTLM authentication. The NTLM authentication method was designed by Microsoft and is used by IIS web servers. It is a proprietary protocol, reversed engineered by clever people and implemented in curl based on their efforts. This kind of behavior should not be endorsed, you should encourage everyone who uses NTLM to switch to a public and documented authentication method instead. Such as Digest.

If you want to enable NTLM for your proxy authentication, then use --proxy-ntlm.

This option requires that the library was built with SSL support. Use -V/--version to see if your curl supports NTLM.

If this option is used several times, the following occurrences make no difference.

-o/--output Write output to instead of stdout. If you are using {} or [] to fetch multiple documents, you can use '#' followed by a number in the specifier. That variable will be replaced with the current string for the URL being fetched. Like in:

curl http://{one,two}.site.com -o "file_#1.txt"

or use several variables like:

curl http://{site,host}.host[1-5].com -o "#1_#2"

You may use this option as many times as you have number of URLs.

See also the --create-dirs option to create the local directories dynamically.

-O/--remote-nameWrite output to a local file named like the remote file we get. (Only the file part of the remote file is used, the path is cut off.)

The remote file name to use for saving is extracted from the given URL, nothing else.

You may use this option as many times as you have number of URLs.

--pass (SSL) Pass phrase for the private key

If this option is used several times, the last one will be used.

--proxy-anyauthTells curl to pick a suitable authentication method when communicating with the given proxy. This will cause an extra request/response round-trip. (Added in 7.13.2)

If this option is used twice, the second will again disable the proxy use-any authentication.

--proxy-basicTells curl to use HTTP Basic authentication when communicating with the given proxy. Use --basic for enabling HTTP Basic with a remote host. Basic is the default authentication method curl uses with proxies.

If this option is used twice, the second will again disable proxy HTTP Basic authentication.

--proxy-digestTells curl to use HTTP Digest authentication when communicating with the given proxy. Use --digest for enabling HTTP Digest with a remote host.

If this option is used twice, the second will again disable proxy HTTP Digest.

--proxy-ntlmTells curl to use HTTP NTLM authentication when communicating with the given proxy. Use --ntlm for enabling NTLM with a remote host.

If this option is used twice, the second will again disable proxy HTTP NTLM.

-p/--proxytunnelWhen an HTTP proxy is used (-x/--proxy), this option will cause non-HTTP protocols to attempt to tunnel through the proxy instead of merely using it to do HTTP-like operations. The tunnel approach is made with the HTTP proxy CONNECT request and requires that the proxy allows direct connect to the remote port number curl wants to tunnel through to.

If this option is used twice, the second will again disable proxy tunnel.

-P/--ftp-port
(FTP) Reverses the initiator/listener roles when connecting with ftp. This switch makes Curl use the PORT command instead of PASV. In practice, PORT tells the server to connect to the client's specified address and port, while PASV asks the server for an ip address and port to connect to.
should be one of:

interface
i.e "eth0" to specify which interface's IP address you
want to use (Unix only)

IP address
i.e "192.168.10.1" to specify exact IP number

host name
i.e "my.host.domain" to specify machine

- make curl pick the same IP address that is already used
for the control connection

If this option is used several times, the last one will be used. Disable the use of PORT with --ftp-pasv. Disable the attempt to use the EPRT command instead of PORT by using --disable-eprt. EPRT is really PORT++.

-qIf used as the first parameter on the command line, the curlrc config file will not be read and used. See the -K/--config for details on the default config file search path.
-Q/--quote (FTP) Send an arbitrary command to the remote FTP server. Quote commands are sent BEFORE the transfer is taking place (just after the initial PWD command to be exact). To make commands take place after a successful transfer, prefix them with a dash '-'. To make commands get sent after libcurl has changed working directory, just before the transfer command(s), prefix the command with '+'. You may specify any amount of commands. If the server returns failure for one of the commands, the entire operation will be aborted. You must send syntactically correct FTP commands as RFC959 defines.

This option can be used multiple times.

--random-file (HTTPS) Specify the path name to file containing what will be considered as random data. The data is used to seed the random engine for SSL connections. See also the --egd-file option.
-r/--range (HTTP/FTP) Retrieve a byte range (i.e a partial document) from a HTTP/1.1 or FTP server. Ranges can be specified in a number of ways.

0-499 specifies the first 500 bytes

500-999 specifies the second 500 bytes

-500 specifies the last 500 bytes

9500- specifies the bytes from offset 9500 and forward

0-0,-1 specifies the first and last byte only(*)(H)

500-700,600-799 specifies 300 bytes from offset 500(H)

100-199,500-599 specifies two separate 100 bytes ranges(*)(H)

(*) = NOTE that this will cause the server to reply with a multipart response!

You should also be aware that many HTTP/1.1 servers do not have this feature enabled, so that when you attempt to get a range, you'll instead get the whole document.

FTP range downloads only support the simple syntax 'start-stop' (optionally with one of the numbers omitted). It depends on the non-RFC command SIZE.

If this option is used several times, the last one will be used.

-R/--remote-timeWhen used, this will make libcurl attempt to figure out the timestamp of the remote file, and if that is available make the local file get that same timestamp.

If this option is used twice, the second time disables this again.

--retry If a transient error is returned when curl tries to perform a transfer, it will retry this number of times before giving up. Setting the number to 0 makes curl do no retries (which is the default). Transient error means either: a timeout, an FTP 5xx response code or an HTTP 5xx response code.

When curl is about to retry a transfer, it will first wait one second and then for all forthcoming retries it will double the waiting time until it reaches 10 minutes which then will be the delay between the rest of the retries. By using --retry-delay you disable this exponential backoff algorithm. See also --retry-max-time to limit the total time allowed for retries. (Added in 7.12.3)

If this option is used multiple times, the last occurrence decide the amount.

--retry-delay Make curl sleep this amount of time between each retry when a transfer has failed with a transient error (it changes the default backoff time algorithm between retries). This option is only interesting if --retry is also used. Setting this delay to zero will make curl use the default backoff time. (Added in 7.12.3)

If this option is used multiple times, the last occurrence decide the amount.

--retry-max-time The retry timer is reset before the first transfer attempt. Retries will be done as usual (see --retry) as long as the timer hasn't reached this given limit. Notice that if the timer hasn't reached the limit, the request will be made and while perform- ing, it may take longer than this given time period. To limit a single request's maximum time, use -m/--max-time. Set this option to zero to not timeout retries. (Added in 7.12.3)

If this option is used multiple times, the last occurrence decide the amount.

-s/--silentSilent mode. Don't show progress meter or error messages. Makes Curl mute.

If this option is used twice, the second will again disable silent mode.

-S/--show-errorWhen used with -s it makes curl show error message if it fails.

If this option is used twice, the second will again disable show error.

--socks4 Use the specified SOCKS4 proxy. If the port number is not specified, it is assumed at port 1080. (Added in 7.15.2)

This option overrides any previous use of -x/--proxy, as they are mutually exclusive.

If this option is used several times, the last one will be used.

--socks5 Use the specified SOCKS5 proxy. If the port number is not specified, it is assumed at port 1080. (Added in 7.11.1)

This option overrides any previous use of -x/--proxy, as they are mutually exclusive.

If this option is used several times, the last one will be used. (This option was previously wrongly documented and used as --socks without the number appended.)

--stderr Redirect all writes to stderr to the specified file instead. If the file name is a plain '-', it is instead written to stdout. This option has no point when you're using a shell with decent redirecting capabilities.

If this option is used several times, the last one will be used.

--tcp-nodelayTurn on the TCP_NODELAY option. See the curl_easy_setopt(3) man page for details about this option. (Added in 7.11.2)

If this option is used several times, each occurrence toggles this on/off.

-t/--telnet-option Pass options to the telnet protocol. Supported options are:

TTYPE= Sets the terminal type.

XDISPLOC= Sets the X display location.

NEW_ENV= Sets an environment variable.

-T/--upload-file This transfers the specified local file to the remote URL. If there is no file part in the specified URL, Curl will append the local file name. NOTE that you must use a trailing / on the last directory to really prove to Curl that there is no file name or curl will think that your last directory name is the remote file name to use. That will most likely cause the upload operation to fail. If this is used on a http(s) server, the PUT command will be used.

Use the file name "-" (a single dash) to use stdin instead of a given file.

You can specify one -T for each URL on the command line. Each -T + URL pair specifies what to upload and to where. curl also sup- ports "globbing" of the -T argument, meaning that you can upload multiple files to a single URL by using the same URL globbing style supported in the URL, like this:

curl -T "{file1,file2}" http://www.uploadtothissite.com

or even

curl -T "img[1-1000].png" ftp://ftp.picturemania.com/upload/

--trace Enables a full trace dump of all incoming and outgoing data, including descriptive information, to the given output file. Use "-" as filename to have the output sent to stdout.

If this option is used several times, the last one will be used.

--trace-ascii Enables a full trace dump of all incoming and outgoing data, including descriptive information, to the given output file. Use "-" as filename to have the output sent to stdout.

This is very similar to --trace, but leaves out the hex part and only shows the ASCII part of the dump. It makes smaller output that might be easier to read for untrained humans.

If this option is used several times, the last one will be used.

--trace-timePrepends a time stamp to each trace or verbose line that curl displays. (Added in 7.14.0)

If this option is used several times, each occurrence will toggle it on/off.

-u/--user Specify user and password to use for server authentication. Overrides -n/--netrc and --netrc-optional.

If you use an SSPI-enabled curl binary and do NTLM autentication, you can force curl to pick up the user name and password from your environment by simply specifying a single colon with this option: "-u :".

If this option is used several times, the last one will be used.

-U/--proxy-user Specify user and password to use for proxy authentication.

If you use an SSPI-enabled curl binary and do NTLM autentication, you can force curl to pick up the user name and password from your environment by simply specifying a single colon with this option: "-U :".

If this option is used several times, the last one will be used.

--url Specify a URL to fetch. This option is mostly handy when you want to specify URL(s) in a config file.

This option may be used any number of times. To control where this URL is written, use the -o/--output or the -O/--remote-name options.

-v/--verboseMakes the fetching more verbose/talkative. Mostly usable for debugging. Lines starting with '>' means "header data" sent by curl, '<' means "header data" received by curl that is hidden in normal cases and lines starting with '*' means additional info provided by curl.

Note that if you only want HTTP headers in the output, -i/--include might be option you're looking for.

If you think this option still doesn't give you enough details, consider using --trace or --trace-ascii instead.

If this option is used twice, the second will again disable verbose.

-w/--write-out Defines what to display on stdout after a completed and successful operation. The format is a string that may contain plain text mixed with any number of variables. The string can be specified as "string", to get read from a particular file you specify it "@filename" and to tell curl to read the format from stdin you write "@-".

The variables present in the output format will be substituted by the value or text that curl thinks fit, as described below. All variables are specified like %{variable_name} and to output a normal % you just write them like %%. You can output a newline by using \n, a carriage return with \r and a tab space with \t.

NOTE: The %-letter is a special letter in the win32-environment, where all occurrences of % must be doubled when using this option.

Available variables are at this point:

url_effective
The URL that was fetched last. This is mostly meaningful if you've told curl to follow location: headers.

http_code
The numerical code that was found in the last retrieved HTTP(S) page.

http_connect
The numerical code that was found in the last response (from a proxy) to a curl CONNECT
request. (Added in 7.12.4)

time_total
The total time, in seconds, that the full operation lasted. The time will be displayed with millisecond resolution.

time_namelookup
The time, in seconds, it took from the start until the name resolving was completed.

time_connect
The time, in seconds, it took from the start until the connect to the remote host (or proxy) was completed.

time_pretransfer
The time, in seconds, it took from the start until the file transfer is just about to begin. This includes all pre-transfer commands and negotiations that are specific to the particular protocol(s) involved.

time_redirect
The time, in seconds, it took for all redirection steps include name lookup, connect, pretransfer and transfer before final transaction was started. time_redirect shows the complete execution time for multiple redirections. (Added in 7.12.3)

time_starttransfer
The time, in seconds, it took from the start until the first byte is just about to be transferred. This includes time_pretransfer and also the time the server needs to calculate the result.

size_download
The total amount of bytes that were downloaded.

size_upload
The total amount of bytes that were uploaded.

size_header
The total amount of bytes of the downloaded headers.

size_request
The total amount of bytes that were sent in the HTTP request.

speed_download
The average download speed that curl measured for the complete download.

speed_upload
The average upload speed that curl measured for the complete upload.

content_type
The Content-Type of the requested document, if there was any.

num_connects
Number of new connects made in the recent transfer. (Added in 7.12.3)

num_redirects
Number of redirects that were followed in the request. (Added in 7.12.3)

ftp_entry_path
The initial path libcurl ended up in when logging on to the remote FTP server. (Added in 7.15.4)

If this option is used several times, the last one will be used.

-x/--proxy Use specified HTTP proxy. If the port number is not specified, it is assumed at port 1080.

This option overrides existing environment variables that sets proxy to use. If there's an environment variable setting a proxy, you can set proxy to "" to override it.

Note that all operations that are performed over a HTTP proxy will transparently be converted to HTTP. It means that certain protocol specific operations might not be available. This is not the case if you can tunnel through the proxy, as done with the -p/--proxytunnel option.

Starting with 7.14.1, the proxy host can be specified the exact same way as the proxy environment variables, include protocol prefix (http://) and embedded user + password.

If this option is used several times, the last one will be used.

-X/--request (HTTP) Specifies a custom request method to use when communicating with the HTTP server. The specified request will be used instead of the method otherwise used (which defaults to GET). Read the HTTP 1.1 specification for details and explanations.

(FTP) Specifies a custom FTP command to use instead of LIST when doing file lists with ftp.

If this option is used several times, the last one will be used.

-y/--speed-time If a download is slower than speed-limit bytes per second during a speed-time period, the download gets aborted. If speed-time is used, the default speed-limit will be 1 unless set with -y.

This option controls transfers and thus will not affect slow connects etc. If this is a concern for you, try the --connect-timeout option.

If this option is used several times, the last one will be used.

-Y/--speed-limit If a download is slower than this given speed, in bytes per second, for speed-time seconds it gets aborted. speed-time is set with -Y and is 30 if not set.

If this option is used several times, the last one will be used.

-z/--time-cond (HTTP) Request a file that has been modified later than the given time and date, or one that has been modified before that time. The date expression can be all sorts of date strings or if it doesn't match any internal ones, it tries to get the time from a given file name instead! See the curl_getdate(3) man pages for date expression details.

Start the date expression with a dash (-) to make it request for a document that is older than the given date/time, default is a document that is newer than the specified date/time.

If this option is used several times, the last one will be used.

--max-redirs Set maximum number of redirection-followings allowed. If -L/--location is used, this option can be used to prevent curl from following redirections "in absurdum". By default, the limit is set to 50 redirections. Set this option to -1 to make it limitless.

If this option is used several times, the last one will be used.

-0/--http1.0(HTTP) Forces curl to issue its requests using HTTP 1.0 instead of using its internally preferred: HTTP 1.1.
-1/--tlsv1(HTTPS) Forces curl to use TSL version 1 when negotiating with a remote TLS server.
-2/--sslv2(HTTPS) Forces curl to use SSL version 2 when negotiating with a remote SSL server.
-3/--sslv3(HTTPS) Forces curl to use SSL version 3 when negotiating with a remote SSL server.
--3p-quote(FTP) Specify arbitrary commands to send to the source server. See the -Q/--quote option for details. (Added in 7.13.0)
--3p-url(FTP) Activates a FTP 3rd party transfer. Specifies the source URL to get a file from, while the "normal" URL will be used as target URL, the file that will be written/created.

Note that not all FTP server allow 3rd party transfers. (Added in 7.13.0)

--3p-user(FTP) Specify user:password for the source URL transfer. (Added in 7.13.0)
-4/--ipv4If libcurl is capable of resolving an address to multiple IP versions (which it is if it is ipv6-capable), this option tells libcurl to resolve names to IPv4 addresses only.
-6/--ipv6If libcurl is capable of resolving an address to multiple IP versions (which it is if it is ipv6-capable), this option tells libcurl to resolve names to IPv6 addresses only.
-#/--progress-barMake curl display progress information as a progress bar instead of the default statistics.

If this option is used twice, the second will again disable the progress bar.

URLThe URL syntax is protocol dependent. You'll find a detailed description in RFC 3986.

You can specify multiple URLs or parts of URLs by writing part sets
within braces as in:

http://site.{one,two,three}.com

or you can get sequences of alphanumeric series by using [] as in:

ftp://ftp.numericals.com/file[1-100].txt
ftp://ftp.numericals.com/file[001-100].txt (with leading zeros)
ftp://ftp.letters.com/file[a-z].txt

No nesting of the sequences is supported at the moment, but you can use several ones next to each other:

http://any.org/archive[1996-1999]/vol[1-4]/part{a,b,c}.html

You can specify any amount of URLs on the command line. They will be fetched in a sequential manner in the specified order.

Since curl 7.15.1 you can also specify step counter for the ranges, so that you can get every Nth number or letter:

http://www.numericals.com/file[1-100:10].txt
http://www.letters.com/file[a-z:2].txt

If you specify URL without protocol:// prefix, curl will attempt to guess what protocol you might want. It will then default to HTTP but try other protocols based on often-used host name prefixes. For example, for host names starting with "ftp." curl will assume you want to speak FTP.

Curl will attempt to re-use connections for multiple file transfers, so that getting many files from the same server will not do multiple connects / handshakes. This improves speed. Of course this is only done on files specified on a single command line and cannot be used between separate curl invokes.

Examples

curl computerhope.com/index.htm > index.htm

The above command would use the curl command to retrieve the index.htm from computerhope and save it as index.htm in the current director

The netstat Command

netstat is a useful tool for checking your network configuration and activity. It is in fact a collection of several tools lumped together. We discuss each of its functions in the following sections.

Displaying the Routing Table

When you invoke netstat with the –r flag, it displays the kernel routing table in the way we've been doing with route. On vstout, it produces:

# netstat -nr

Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
127.0.0.1 * 255.255.255.255 UH 0 0 0 lo
172.16.1.0 * 255.255.255.0 U 0 0 0 eth0
172.16.2.0 172.16.1.1 255.255.255.0 UG 0 0 0 eth0

The –n option makes netstat print addresses as dotted quad IP numbers rather than the symbolic host and network names. This option is especially useful when you want to avoid address lookups over the network (e.g., to a DNS or NIS server).

The second column of netstat 's output shows the gateway to which the routing entry points. If no gateway is used, an asterisk is printed instead. The third column shows the “generality” of the route, i.e., the network mask for this route. When given an IP address to find a suitable route for, the kernel steps through each of the routing table entries, taking the bitwise AND of the address and the genmask before comparing it to the target of the route.

The fourth column displays the following flags that describe the route:

G

The route uses a gateway.

U

The interface to be used is up.

H

Only a single host can be reached through the route. For example, this is the case for the loopback entry 127.0.0.1.

D

This route is dynamically created. It is set if the table entry has been generated by a routing daemon like gated or by an ICMP redirect message (see the section Section 2.5” in Chapter 2).

M

This route is set if the table entry was modified by an ICMP redirect message.
!

The route is a reject route and datagrams will be dropped.

The next three columns show the MSS, Window and irtt that will be applied to TCP connections established via this route. The MSS is the Maximum Segment Size and is the size of the largest datagram the kernel will construct for transmission via this route. The Window is the maximum amount of data the system will accept in a single burst from a remote host. The acronym irtt stands for “initial round trip time.” The TCP protocol ensures that data is reliably delivered between hosts by retransmitting a datagram if it has been lost. The TCP protocol keeps a running count of how long it takes for a datagram to be delivered to the remote end, and an acknowledgement to be received so that it knows how long to wait before assuming a datagram needs to retransmitted; this process is called the round-trip time. The initial round-trip time is the value that the TCP protocol will use when a connection is first established. For most network types, the default value is okay, but for some slow networks, notably certain types of amateur packet radio networks, the time is too short and causes unnecessary retransmission. The irtt value can be set using the route command. Values of zero in these fields mean that the default is being used.

Finally, the last field displays the network interface that this route will use.
Displaying Interface Statistics

When invoked with the –i flag, netstat displays statistics for the network interfaces currently configured. If the –a option is also given, it prints all interfaces present in the kernel, not only those that have been configured currently. On vstout, the output from netstat will look like this:

# netstat -i
Kernel Interface table
Iface MTU Met RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flags
lo 0 0 3185 0 0 0 3185 0 0 0 BLRU
eth0 1500 0 972633 17 20 120 628711 217 0 0 BRU

The MTU and Met fields show the current MTU and metric values for that interface. The RX and TX columns show how many packets have been received or transmitted error-free (RX-OK/TX-OK) or damaged (RX-ERR/TX-ERR); how many were dropped (RX-DRP/TX-DRP); and how many were lost because of an overrun (RX-OVR/TX-OVR).

The last column shows the flags that have been set for this interface. These characters are one-character versions of the long flag names that are printed when you display the interface configuration with ifconfig:

B

A broadcast address has been set.
L

This interface is a loopback device.
M

All packets are received (promiscuous mode).
O

ARP is turned off for this interface.
P

This is a point-to-point connection.
R

Interface is running.
U

Interface is up.

Displaying Connections

netstat supports a set of options to display active or passive sockets. The options –t, –u, –w, and –x show active TCP, UDP, RAW, or Unix socket connections. If you provide the –a flag in addition, sockets that are waiting for a connection (i.e., listening) are displayed as well. This display will give you a list of all servers that are currently running on your system.

Invoking netstat -ta on vlager produces this output:

$ netstat -ta
Active Internet Connections
Proto Recv-Q Send-Q Local Address Foreign Address (State)
tcp 0 0 *:domain *:* LISTEN
tcp 0 0 *:time *:* LISTEN
tcp 0 0 *:smtp *:* LISTEN
tcp 0 0 vlager:smtp vstout:1040 ESTABLISHED
tcp 0 0 *:telnet *:* LISTEN
tcp 0 0 localhost:1046 vbardolino:telnet ESTABLISHED
tcp 0 0 *:chargen *:* LISTEN
tcp 0 0 *:daytime *:* LISTEN
tcp 0 0 *:discard *:* LISTEN
tcp 0 0 *:echo *:* LISTEN
tcp 0 0 *:shell *:* LISTEN
tcp 0 0 *:login *:* LISTEN

This output shows most servers simply waiting for an incoming connection. However, the fourth line shows an incoming SMTP connection from vstout, and the sixth line tells you there is an outgoing telnet connection to vbardolino.[1]

Using the –a flag by itself will display all sockets from all families.

Thursday, January 13, 2011

Checkpoint - Log File Corrupted

When log files get corrupted and we get the following error in the SmartView Tracker : Failed to read record number …

To repare the log file we need to know the log file name and then from the CLI on the CLM/CMA or from the Smartcenter andn then use the following command :

[Expert@mlm]# fw repairlog

Changing MTU in SPLAT

Sometimes you need to adjust the MTU manually (ipsec or dsl overhead can cause issues)

#ifconfig mtu

If you need your change to be reboot persitent :

add the ifconfig mtu line in /etc/rc.d/rc.local

Configuring NTP on SPLAT

ntp -n

interval in seconds on R65

example : ntp -n 600 84.45.68.23 141.89.226.53

check the config :

#Expert@fw1]# cpd_sched_config print
Task: “av_http_server”
Command: av_http_server
Arguments: -j -f /opt/CPsuite-R65/fw1/conf/av_http_server.conf
Interval: 300
Active: true
RunAtStart: true
Task: “RotateLogs”
Command: /sbin/cp_logrotate
Arguments:
Interval: 100
Active: true
RunAtStart: true
Task: “NtpPoll”
Command: /usr/sbin/ntpdate -s -f 84.45.68.23 141.89.226.53
Arguments:
Interval: 0
Active: true
RunAtStart: true
[Expert@kukuw-lb-fw1]# ntp -n 60 84.45.68.23 141.89.226.53
[Expert@kukuw-lb-fw1]# cpd_sched_config print
Task: “av_http_server”
Command: av_http_server
Arguments: -j -f /opt/CPsuite-R65/fw1/conf/av_http_server.conf
Interval: 300
Active: true
RunAtStart: true
Task: “RotateLogs”
Command: /sbin/cp_logrotate
Arguments:
Interval: 100
Active: true
RunAtStart: true
Task: “NtpPoll”
Command: /usr/sbin/ntpdate -s -f 84.45.68.23 141.89.226.53
Arguments:
Interval: 60
Active: true
RunAtStart: true

NTP on SPLAT can be configured from the Web interface as well

ClusterXL : ping clusterxl and members interfaces simultaneously

By default pinging the cluster and members interfaces at the same time wont work or will randomly fail.

If you cant reboot your firewall(s), you just run :
#fw ctl set int fw_allow_simultaneous_ping 1

If you need the change to be rreboot persistent, run the following commands on enforcements modules :
#touch $FWDIR/boot/modules/fwkern.conf (if the file doesnt exist already)
#echo “fw_allow_simultaneous_ping=1″ >> $FWDIR/boot/modules/fwkern.conf
#reboot

Excluded interfaces

All interfaces that are not part from ClusterXL should be defined in :

$FWDIR/conf/discntd.if

Manual failover in clusterXL

[Expert@FW-1]#
[Expert@FW-1]# clusterXL_admin down
Setting member to administratively down state ...
Member current state is Down
[Expert@FW-1]# clusterXL_admin up
Setting member to normal operation ...
Member current state is Standby
[Expert@FW-1]#
[Expert@FW-1]#
[Expert@FW-1]#
[Expert@FW-1]#
[Expert@FW-1]# cphaprob state

Cluster Mode: New High Availability (Active Up)


Number Unique Address Assigned Load State

1 192.168.252.253 100% Active
2 (local) 192.168.252.254 0% Standby

Cluster Control Protocol mode

Switching to broadcast mode : cphaconf set_ccp broadcast

Switching to multicast mode : cphaconf set_ccp multicast

Default settings in $FWDIR/boot/ha_boot.conf are
ha_installed 1
ccp_mode broadcast

When using ClusterXL udp port 8116 is sent on all the interfaces of the gateway cluster members (except those define in $FWDIR/conf/discntd.if). UDP port 8116 is necessary for cluster health check. Checkpoint High-Availability is located between vpn-1/firewall-1 kernel and the network cards. This is the reason why security policy cant block synchronization data. So we dont need to create explicite rule in the dashboard for it. This is also the reason CCP packets should be captured via tcpdump.

Restarting ClusterXL synchronization

To perform a full sync, run
# fw ctl setsync off

then run
#fw ctl setsync start

check logs in $FWDIR/log/fwd.elg

Cluster XL freeze mechanism

Policy installation under heavy load can cause a failover

There is a freeze mechanism, you can implement it as follow :

fw ctl set int fwha_freeze_state_machine_timeout 30 (This command should be run on both cluster members)

To survive a reboot, add the fwha_freeze_state_machine_timeout=30 command in fwkern.conf

Two clusters sharing the same VLAN

This assume you using CCP (cluster control protocol) in broadcast mode

Can be verified while running : #cphaprob -a if

On cluster A
# fw ctl set int fwha_mac_magic 250
# fw ctl set int fwha_mac_forward_magic 251

On cluster B:
# fw ctl set int fwha_mac_magic 240
# fw ctl set int fwha_mac_forward_magic 241

not sure the above commands are “reboot persitent” (I did not test it) so the change can be implemented in : $FWDIR/boot/modules/fwkern.conf

example :

[Expert@toto]# cat /opt/CPsuite-R65/fw1/boot/modules/fwkern.conf
fwha_mac_magic=250
fwha_mac_forward_magic=251

SPLAT : Snapshot and Revert

The SecurePlatform Image and configuration can be saved and reverted with the snapshot and revert commands. These commands can be run in Standard and Expert Modes, and can use a TFTP or SCP server to store snapshots (locally, if necessary).


  • snapshot and revert command syntax can be displayed using "-h" or the "--help" switch.
    Example:

    snapshot -h

    [-h] [-d] [[--tftp ServerIP Filename]|
    [--scp ServerIP Username Password Filename] |
    [--file Filename]]


    Where switch:

    -d (Shows debug messages)
    -h, --help (Shows this help information)
    -t, --tftp (Transfers backup package to TFTP server)
    -s, --scp (Transfers backup package to SCP server)
    -f, --file (Specifies local backup-package filename)




  • Using the snapshot command without any switch displays a list of choices.

    Procedure:

    1. In Standard mode, type snapshot.


    2. Choose one of the following:


    3. L Create new local Snapshot Image
      T Create new snapshot Image on TFTP server
      S Create new snapshot Image on SCP server
      R Remove local Snapshot Image
      Q Quit

    4. Select a choice. The following line appears:


    5. Creating the Snapshot Image. This can take up to 10 minutes...

    6. The new snapshot output file can be found at /var/CPsnapshot/snapshots, by typing the ls command in Expert Mode:
      Example:

      [Expert@cpmodule]#ls /var/CPsnapshot/snapshots



  • Using the revertcommand without any switch displays a list of choices.

    Procedure:

    1. Type revert to get the following display:


      L Revert to local Snapshot Image
      T Revert to Snapshot Image from TFTP
      S Revert to Snapshot Image from SCP
      R Remove local Snapshot Image
      Q Quit


    2. Type L



    3. Revert to:
      [1] new-snap.tgz
      [2] old-snap.tgz
      [Q] Quit


    4. Select from the following:


      Do you want to save snapshot image before revert?

      L Create new local Snapshot Image
      T Create new Snapshot Image on TFTP
      S Create new Snapshot Image on SCP server
      O Override current Snapshot Image (/var/CPsnapshot/snapshots/old-snap.tgz)
      R Remove local Snapshot Image
      Q Quit

Note:
After the revert process has finished, SecurePlatform reboots.

SIC - Reinitialize between the Security Gateway and the SmartCenter server

Symptoms

* Error when trying to initialize SIC with the Security Gateway.
* Error: "Failed to connect the module".

Solution
-

To reinitialize SIC (Secure Internal Communication) between the Security Gateway and the SmartCenter server, proceed as follows:

Scenario 1: Security Gateway is installed on a UNIX machine

1. On the Security Gateway, run cpconfig and reinitialize the Secure Internal Communication as follows.

2. In the Configuration Options screen, type in the &ltSecure Internal Communication Option number> and press 'Enter'.

3. In the Configuring Secure Internal Communication screen, when prompted to re-initialize communication, type in "y" and press 'Enter'.

4. The next screen verifies that you want to reset Secure Internal Communication now. Type in "y" and press 'Enter'.

5. Enter Activation Key: ****** (For example, abc123)

6. Again Activation Key: ****** (For example, abc123)

7. The Secure Internal Communication Successfully Initialized screen is displayed. Type in the &ltExit Option number> and press 'Enter'.

8. The next screen prompts you to restart ALL Check Point modules (performing cpstop & cpstart) in order to activate the changes you have made. Type in "y" and press 'Enter'.
9. In SmartDashboard, select 'Manage > Network Objects'.

10. In Network Objects Manager, select the Security Gateway network object.

11. Click 'Edit'.

12. In the dialog box, select General Properties from the left pane.

13. In General Properties, click 'Communication' in the Secure Internal Communication section.

14. In the Communication dialog box, click 'Reset'. The dialog box will be seen with the following message:

For the reset operation to be complete, you must also reset
the module in the configuration tool. No communication will
be possible until you reset and re-initialize the communication
properly.
Are you sure you want to reset?


15. Click 'Yes'.
16. The dialog box displays the following message:

Reset is done.
Please re-install the Security Policy in order to update the CRL list.
You must install the Security Policy to ALL Modules.

Click 'OK'.

17. In the Communication dialog box, enter the one time password (abc123) in the Activation Key field.

18. Re-enter the one time password (abc123) in the Confirm Activation Key field.

Note: The one time password, entered in the Activation Key field and Confirm Activation Key field, needs to be the same one time password that has been set as the Activation Key on the Security Gateway.


19. Click 'Initialize'.

20. Verify the Trust state field is changed to "Trust established".

21. To test Secure Internal Communication status, click 'Test SIC status'.

22. Verify that the message in the SIC Status dialog box is similar to:

SIC Status for fw_module: Communicating


23. Click 'Close' in the SIC Status dialog box.

24. Click 'Close' in the Communication dialog box.

25. Click 'OK' in the Check Point Gateway dialog box.

26. Install policy.

Scenario 2: Security Gateway is installed on a Windows machine

1. On the Security Gateway, access cpconfig.

2. In the Check Point Configuration Tool dialog box, select the Secure Internal Communication tab.

3. In the Secure Internal Communication tab, click 'Reset' in the Initialization section.

The Warning dialog box will display the following message:

For the reset operation to be complete you must also reset the module in the Policy Editor.
No communication will be possible until you reset and re-initialize the communication properly.

Note: The Secure Internal Communication will be reset now.

Are you sure you want to reset?


4. Click 'Yes'.

5. In the Secure Internal Communication tab, enter the one time password (For example, def456) in the Activation Key field.

6. Re-enter the one time password (def456) in the Confirm Activation Key field.
7. Click 'OK' in the Check Point Configuration Tool.

The cpconfig dialog box will be seen with the following message:

You have changed the system configuration.
It is recommended to restart Check Point services.

Do you wish to restart the services now?


8. Click 'Yes'.


9. In SmartDashboard, select 'Manage > Network Objects'.

10. In the Network Objects Manager, select the Security Gateway network object.

11. Click 'Edit'.

12. In the Check Point Gateway dialog box, select General Properties from the left pane.

13. In General Properties, click 'Communication' in the Secure Internal Communication section.

14. In the Communication dialog box, click 'Reset'.

The dialog box displays the following message:

For the reset operation to be complete, you must also reset the module in the configuration tool. No communication will be possible until you reset and re-initialize the communication properly.
Are you sure you want to reset?


15. Click 'Yes'. The dialog box will display the following message:

Reset is done.
Please re-install the Security Policy in order to update the CRL list.
You must install the Security Policy to ALL Modules.


16. Click 'OK'.
17. In the Communication dialog box, enter the one time password (def456) in the Activation Key field.

18. Re-enter the one time password (def456) in the Confirm Activation Key field.

Note: The one time password, entered in the Activation Key field and Confirm Activation Key field, needs to be the same password set as the Activation Key on the Security Gateway.


19. Click 'Initialize'.

20. Verify that the message in the Trust state field is changed to "Trust established".

21. To test Secure Internal Communication status, click 'Test SIC status'.

22. Verify message in the SIC Status dialog box is similar to:

SIC Status for fw_module: Communicating


23. Click 'Close' in the SIC Status dialog box.

24. Click 'Close' in the Communication dialog box.

25. Click 'OK' in the Check Point Gateway dialog box.

26. Install policy.

Wednesday, January 12, 2011

Reason for Firewall Drop : fw ctl zdebug drop

One of my favorite troubleshooting command is "fw ctl zdebug drop"

This will show the realtime logs.. No need of worrying where is my Tracker for a real time log..

The output is very nice, shows the reason for the drop and can easily be filtered with the grep command for IP addresses:

fw_log_drop: Packet proto=17 10.255.253.21:20031 -> 10.255.253.255:20031 dropped by fw_antispoof_log Reason: Address spoofing

fw_log_drop: Packet proto=17 192.243.100.205:58999 -> 224.0.0.1:9996 dropped by fw_handle_first_packet Reason: Rulebase drop - rule 243

fw_log_drop: Packet proto=1 10.68.111.2:1281 -> 10.68.111.5:1669 dropped by fw_icmp_stateless_checks Reason: ICMP redirect packets are not allowed

fw_log_drop: Packet proto=6 192.243.119.238:80 -> 91.96.46.174:49543 dropped by fw_first_packet_state_checks Reason: First packet isn't SYN

Since this is realtime debug output, you need to have live traffic through the firewall to see if a packet is dropped. When you try to investigate the reason for a drop of an older connection, you have to go the SmartView Tracker.

One more example....
[Expert@FW-1]# fw ctl zdebug drop | grep 3.212.227.23
;[cpu_2];[fw_1];fw_log_drop: Packet proto=6 3.212.227.23:51459 -> 3.245.237.213:443 dropped by fw_handle_first_packet Reason: Rulebase drop - rule 38;
;[cpu_2];[fw_1];fw_log_drop: Packet proto=6 3.212.227.23:51459 -> 3.245.237.213:443 dropped by fw_handle_first_packet Reason: Rulebase drop - rule 38;
;[cpu_2];[fw_1];fw_log_drop: Packet proto=6 3.212.227.23:51459 -> 3.245.237.213:443 dropped by fw_handle_first_packet Reason: Rulebase drop - rule 38;


=======================================================

To check why traffic is being silently dropped by a firewall, simultaneously run the following:

1. FW MONITOR to gather all traffic between Host A and Host B. The syntax would be:

fw monitor -e "accept src=n.n.n.n or dst=x.x.x.x or src=y.y.y.y or dst=y.y.y.y;" -o fwmon.cap &

where n.n.n.n is the IP address of Host A, and y.y.y.y is the IP address of Host B. The '&' sign will put fw monitor in the background.

2. Start the kernel debugging for drop activities:

fw ctl zdebug drop > fwctl.txt

Now replicate the problem to gather the debugs.

When finished, press Ctrl+C to break the 'fw ctl zdebug' command and then type 'fg' to bring back the 'fw monitor' to the foreground.

Enter Ctrl+C again to cancel the fw monitor.

Analyse the output files fwmon.cap and fwctl.txt to determine what is being dropped and why it is being dropped

============================================================

Checkpoint Ports

TCP Port 256 is used for three important things:

Exchange of CA and DH keys in FWZ and SKIP encryption between two FireWall-1 Management Consoles SecuRemote build 4005 and earlier uses this port to fetch the network topology and encryption keys from a FireWall-1 Management Console
When instaling a policy, the management console uses this port to push the policy to the remote firewall.

TCP Port 257 (FW1_log) is used for logging purposes.

TCP Port 258 is used by the fwpolicy remote GUI. FireWall-1 will only listen to this port on a management console.

TCP Port 259 is used for Client Authentication via telnet. FireWall-1 will only listen to this port on a firewall module.

UDP Port 259 is used in FWZ encryption to manage the encrypted session (SecuRemote and FireWall-1 to FireWall-1 VPNs).

UDP Port 260 and UDP Port 161 are used for the SNMP daemon that Check Point FireWall-1 Provides.

TCP Port 262 is used by netsod, which is the Single Sign-on Daemon. This can be disabled by commenting out the line that contains netsod in $FWDIR/conf/fwauthd.conf.

TCP Port 264 is used for Secure Client (SecuRemote) build 4100 and later to fetch network topology and encryption keys from a FireWall-1 Management Console. FireWall-1 will only listen to this port on a management console.

UDP Port 500 is used for ISAKMP key exchange between firewalls or between a firewall and a host running Secure Client. FireWall-1 will only listen to this port on a firewall module.

TCP Port 900 is used by FireWall-1's HTTP Client Authentication mechanism. This can be disabled by commenting the appropriate line in $FWDIR/bin/fwauthd.conf.

TCP Port 4532 is used for the Session Auth agent, asessiond.TCP Ports above 1024, other than the ones listed below, are generally any Security Servers that are active. The actual ports used by these servers will vary. 1024 is the lowest & 65535 is the highest port a Security Server process will use for a connection. If you wish to minimize the number of ports listening, comment out the appropriate lines from $FWDIR/conf/fwauthd.conf. Note that if you disable a security server in this fashion, you can not use its capabilities (like content security or authentication), so make sure you only do it for the security servers you know you are not using.

TCP Port 18181 is used for CVP (Content Vectoring Protocol, for anti-virus scanning). FireWall-1 will not listen on this port.TCP Port 18182 is used for UFP (URL Filtering Protocol, for WebSense and the like). FireWall-1 will not listen on this port.

TCP Port 18183 is used for SAM (Suspicious Activity Monitoring, for intrusion detection). If you do not use these features, you can comment out the appropriate lines in $FWDIR/conf/fwopsec.conf.

TCP Port 18184 is used for Log Export API (lea). If you do not use these features, you can comment out the appropriate lines in $FWDIR/conf/fwopsec.conf.

TCP Port 18186 (FW1_omi-sic) is used for Secure Internal Communications (SIC) between OPSEC certified products and a NG FireWall module.

TCP Port 18190 (CPMI) is used by the FireWall Management process (FWM) to listen for NG Management Clients attempting to connect to the management module.

TCP Port 18191 (CPD) is used by the CPD process for communications such as policy installation, certificate revocation, and status queries.

TCP Port 18192 (CPD_amon) is used by the CPD process FireWall Application Monitoring.

TCP Port 18196 is used for CPEPS which is part of User Monitor.

TCP Port 18207 is used by polsrvd, which is the Single Sign-on Daemon. This can be disabled by commenting out the line that contains polsrvd in $FWDIR/conf/fwauthd.conf.

TCP Port 18210 (FW1_ica_pull): The CPD process, on the management module, is listening on
TCP port 18210 for certificates to be "pulled" by a FireWall module from a management module.

TCP Port 18211 (FW1_ica_push): The Check Point Daemon (CPD) process, running on the FireWall module, listens on TCP port 18211 for certificate creation and for the "push" of the certificate to the FireWall module from the management module.

Should you make any changes above, the 'fwd' process will need to be restarted as follows:

nokia[admin]# fw kill fwdnokia

[admin]# fwd `cat $FWDIR/conf/masters`