For a while now I have had an issue using my favorite IDE's (eclipse, visual studio). Every time I hit ctrl-space (auto complete) it would switch to Chinese characters.
It turns out that ctrl-space is windows hot-key used to switch keyboards. If you have multiple keyboards installed (or say installed by default by your manufacturer who pre-loaded windows for you), the ctrl-space command would be intercepted by windows, and you would switch keyboards to the new language. I don't know if this has anything to do with the fact that the Chinese keyboard was listed first in my properties.
In any event, I just stumbled upon this SO question and it seems to have solved my issues.
http://stackoverflow.com/questions/179119/how-to-prevent-windows-xp-from-stealing-my-input-ctrl-space-which-is-meant-for-em
Wednesday, January 19, 2011
Book Review: Release It!
Having spent most of my time reading purely technical books to solve specific problems, it was refreshing to read a higher-level book that talks about good programming design. Release It! is by far one of the best books I have read and really opened my mind to areas of program design that I had only started to touch the surface on.
As it is one of the best forms of persuasion, I really enjoyed the story-telling in the book. I felt that it gave the main content of this book depth and realism. It is good to know that everyone makes these types of mistakes.
The main contents of this book are broken up into four distinct areas. The first two areas describe stability and capacity patterns and anti-patterns. It was interesting to read the unique challenges that large scale applications with millions of users face. Things that we take for granted in smaller websites just wouldn't do in larger ones. For example, say you want to stop a bot from scanning your site. An easy way to do this is to force all the bots to establish a session before granting them access to content. Most (read all) bots cannot hold a session since they are there purely for scrapping purposes. The problem with this is that each session that is created because of a hit from a bot is stored somewhere in memory on a server. Combine that with thousands of users trying to use your website at the same time, and we have a problem. On small sites, you could probably get away with the memory being tied up for a specified timeout period, but on larger sites, you will not be so lucky.
The stability patterns described in the book are as follows:
1) Use timeouts
2) Circuit Breaker
3) Bulkheads
4) Steady State
5) Fail Fast
6) Handshaking
7) Test Harness
8) Decoupling Middleware
The capacity patterns:
1) Pool connections
2) Use Caching carefully
3) Precompute Content
4) Tune the garbage collector
He then goes on to describe some general design issues that has come up in his vast experience. He talks a lot about developing SLA's with the business (or client) and then using those to help define the level of redundancy that a particular application needs. He also talks a lot about administrating interfaces. If you don't build applications that can be maintained, guess what, they won't be. In the last section he talks about some coding designs that should be incorporated into any development process. Those things include concepts like transparency.
It is interesting to be in this industry. You see many projects directed towards delivering results with very little design up front. The fact of the matter is, companies would probably save a lot of money if they spent more on the design time. You have to understand that designs can be reused. Chances are you are building applications a certain way because they fit your business style. With that in mind, other applications that you develop will also meet that same style. A bit of planning and design a head of time will have huge rewards in the long run and benefit projects down the line. Another problem that I see is that consulting firms are hired based on price, but not on quality. I sometimes feel that business have a hard time separating the good from the bad. Did the project meet the requirements? maybe.... Did the project meet the budget? yes.... CHECK! Little do they know that the application was not built in an extensible way. It was not built with any future considerations. It will need to be replaced in 3 years rather than 5.
In any event, I recommend that all developers read this book, and, most importantly, keep the concepts that he talks about in this book in the back of your mind when developing any application. You will probably find that your application handles failure better, and delivers a better overall customer experience.
As it is one of the best forms of persuasion, I really enjoyed the story-telling in the book. I felt that it gave the main content of this book depth and realism. It is good to know that everyone makes these types of mistakes.
The main contents of this book are broken up into four distinct areas. The first two areas describe stability and capacity patterns and anti-patterns. It was interesting to read the unique challenges that large scale applications with millions of users face. Things that we take for granted in smaller websites just wouldn't do in larger ones. For example, say you want to stop a bot from scanning your site. An easy way to do this is to force all the bots to establish a session before granting them access to content. Most (read all) bots cannot hold a session since they are there purely for scrapping purposes. The problem with this is that each session that is created because of a hit from a bot is stored somewhere in memory on a server. Combine that with thousands of users trying to use your website at the same time, and we have a problem. On small sites, you could probably get away with the memory being tied up for a specified timeout period, but on larger sites, you will not be so lucky.
The stability patterns described in the book are as follows:
1) Use timeouts
2) Circuit Breaker
3) Bulkheads
4) Steady State
5) Fail Fast
6) Handshaking
7) Test Harness
8) Decoupling Middleware
The capacity patterns:
1) Pool connections
2) Use Caching carefully
3) Precompute Content
4) Tune the garbage collector
He then goes on to describe some general design issues that has come up in his vast experience. He talks a lot about developing SLA's with the business (or client) and then using those to help define the level of redundancy that a particular application needs. He also talks a lot about administrating interfaces. If you don't build applications that can be maintained, guess what, they won't be. In the last section he talks about some coding designs that should be incorporated into any development process. Those things include concepts like transparency.
It is interesting to be in this industry. You see many projects directed towards delivering results with very little design up front. The fact of the matter is, companies would probably save a lot of money if they spent more on the design time. You have to understand that designs can be reused. Chances are you are building applications a certain way because they fit your business style. With that in mind, other applications that you develop will also meet that same style. A bit of planning and design a head of time will have huge rewards in the long run and benefit projects down the line. Another problem that I see is that consulting firms are hired based on price, but not on quality. I sometimes feel that business have a hard time separating the good from the bad. Did the project meet the requirements? maybe.... Did the project meet the budget? yes.... CHECK! Little do they know that the application was not built in an extensible way. It was not built with any future considerations. It will need to be replaced in 3 years rather than 5.
In any event, I recommend that all developers read this book, and, most importantly, keep the concepts that he talks about in this book in the back of your mind when developing any application. You will probably find that your application handles failure better, and delivers a better overall customer experience.
Saturday, January 15, 2011
Top 10 mistakes made in behavior change
So I was reading hacker news and found the following link.
http://www.slideshare.net/captology/stanford-6401325
I suggest that everyone read it. Although it is just a point form slide show, it really hits the spot with why I think many people (including myself) fail at certain goals. There are three points that I really want to touch on.
1) Ignoring how environment shapes behavior
2) Trying to stop old behaviors instead of creating new ones
3) Underestimating the power of triggers
I will start with a little story. My wife and I had set a goal of watching less TV. This goal has obvious benefits. For some reason, tho, we could never seem to get away from the TV. The problem was this. We lived in a condo, and as soon as you walked in, you were basically right at the TV. It was hard to "sit on the couch" for a few minutes after work without turning on the TV. I finally decided one day to move the TV from the "living room" to the "second bedroom". Instantly, we have a nice, accessible area to just hang out, without the pressure of the TV also being in the same room. I found it much easier to break away from wasting evenings completely watching TV. It was great, and felt liberating. It could not have happened if I had not realized how the environment I was in was affecting me.
Number 2 above really speaks to making positive, rather than negative goals. For example. The goal to want to stop smoking is really a negative goal. You could, instead, set a goal to live a healthier life. Of course that goal would have to be more specific than that, otherwise it would just be meaningless. I really like they way the slideshow summarized this point, however.
Number 3 is really about understanding why you fail. Lets say you eat junk food when you are stressed. You make a goal to eat less junk food (or, alternatively, eat healthier) but sometimes you digress. It is important to understand the "failure" and try to figure out what triggered you to digress. Without understanding that, you can never get to the root of the problem. The problem is not that you eat too much junk food, it is that you are too stressed in your life.
http://www.slideshare.net/captology/stanford-6401325
I suggest that everyone read it. Although it is just a point form slide show, it really hits the spot with why I think many people (including myself) fail at certain goals. There are three points that I really want to touch on.
1) Ignoring how environment shapes behavior
2) Trying to stop old behaviors instead of creating new ones
3) Underestimating the power of triggers
I will start with a little story. My wife and I had set a goal of watching less TV. This goal has obvious benefits. For some reason, tho, we could never seem to get away from the TV. The problem was this. We lived in a condo, and as soon as you walked in, you were basically right at the TV. It was hard to "sit on the couch" for a few minutes after work without turning on the TV. I finally decided one day to move the TV from the "living room" to the "second bedroom". Instantly, we have a nice, accessible area to just hang out, without the pressure of the TV also being in the same room. I found it much easier to break away from wasting evenings completely watching TV. It was great, and felt liberating. It could not have happened if I had not realized how the environment I was in was affecting me.
Number 2 above really speaks to making positive, rather than negative goals. For example. The goal to want to stop smoking is really a negative goal. You could, instead, set a goal to live a healthier life. Of course that goal would have to be more specific than that, otherwise it would just be meaningless. I really like they way the slideshow summarized this point, however.
Number 3 is really about understanding why you fail. Lets say you eat junk food when you are stressed. You make a goal to eat less junk food (or, alternatively, eat healthier) but sometimes you digress. It is important to understand the "failure" and try to figure out what triggered you to digress. Without understanding that, you can never get to the root of the problem. The problem is not that you eat too much junk food, it is that you are too stressed in your life.
Thursday, January 13, 2011
mod_security, apache httpd, glassfish - Part 4
Part 4, mod_sec install.
One dependency that I missed downloading earlier was libXML2. This is a dependency for mod_security. If you wish, you can also install lua and curl. Lua is needed if you plan to write your own rules and want to use the new lua engine. I'm not really planning on writing rules, the base rules are pretty good. As for curl, it is only needed if you want to send logs to a central repo like loglogics. Since I don't have that kind of infrastructure setup, I won't worry about that in these posts. If you wish to install those, simple follow the install instructions and then edit the configure to point to the locations in which you have installed the dependencies.
For now, navigate over to http://xmlsoft.org/downloads.html and download libXML2.
Now we can install mod_security
When you untared the mod_security files, there was a rules directory. We are going to copy that directory to /path/to/httpd/conf/rules. Next, you are going to copy the modsecurity_crs_10_config.conf.example to the same file name minus the example part. The purpose of this series of articles is not to go through an indepth setup of mod_security. Just enough to get it working. You are going to go into the file you just copied and make the following change.
This is basically going to turn the rules engine on. You may want to put it on detection only if you are testing a legacy app. We are going to create another file by the name of modsecurity_crs_10_global_config.conf. In this file, we are going to basically setup some more mod_security rules.
Basically this just sets the server to identify itself as IIS 6. This is a little "security through obscurity". Hopefully a few script kiddies will get deterred by this.
We need to add the following to our httpd.conf
We need to now actually apply these rules to our site that we have just created. In our site.conf, we should add the following.
Include conf/rules/modsecurity_crs_10_config.conf
Include conf/rules/base_rules/*.conf
Include conf/rules/optional_rules/*.conf
When you startup your httpd now, you may get some "syntax" errors inside the mod_security configuration files. I just go in an delete/comment out the line in question. This is obviously okay if you are not using the technology the line pertains to. For example, you can probably safely comment out any php rules if you are not using php in your website. One in particular is about a data file missing for comment spam. You can go and get it (say google) or you can just deal with the line that is causing the issue.
After you startup apache, you will notice that the modsec_debug.log is getting populated. Feel free to test out mod_security. You can either use a tool like w3af, or just try putting sql injection as a parameter. You should see mod_security block it.
One dependency that I missed downloading earlier was libXML2. This is a dependency for mod_security. If you wish, you can also install lua and curl. Lua is needed if you plan to write your own rules and want to use the new lua engine. I'm not really planning on writing rules, the base rules are pretty good. As for curl, it is only needed if you want to send logs to a central repo like loglogics. Since I don't have that kind of infrastructure setup, I won't worry about that in these posts. If you wish to install those, simple follow the install instructions and then edit the configure to point to the locations in which you have installed the dependencies.
For now, navigate over to http://xmlsoft.org/downloads.html and download libXML2.
./configure --prefix=/path/to/deps/ --enable-shared
Now we can install mod_security
CC="gcc -m64" ./configure --prefix=/path/to/deps/ --with apxs=/path/to/httpd/bin/apxs --with-pcre=/path/to/deps/bin/pcre-config --with-apr=/path/to/deps/bin/apr-1-config --with-apu=/path/to/deps/bin/apu-1-config --with-libxml=/path/to/deps/bin/xml2-config
When you untared the mod_security files, there was a rules directory. We are going to copy that directory to /path/to/httpd/conf/rules. Next, you are going to copy the modsecurity_crs_10_config.conf.example to the same file name minus the example part. The purpose of this series of articles is not to go through an indepth setup of mod_security. Just enough to get it working. You are going to go into the file you just copied and make the following change.
# # Review your SecRuleEngine settings. If you want to # allow blocking, then set it to On however check your SecDefaultAction setting # to ensure that it is set appropriately. # #SecRuleEngine DetectionOnly SecRuleEngine On
This is basically going to turn the rules engine on. You may want to put it on detection only if you are testing a legacy app. We are going to create another file by the name of modsecurity_crs_10_global_config.conf. In this file, we are going to basically setup some more mod_security rules.
SecServerSignature "Microsoft-IIS/6.0" SecDebugLog /cardlock/httpd/logs/modsec_debug.log SecDebugLogLevel 3
Basically this just sets the server to identify itself as IIS 6. This is a little "security through obscurity". Hopefully a few script kiddies will get deterred by this.
We need to add the following to our httpd.conf
<ifmodule security2_module> Include conf/rules/modsecurity_crs_10_global_config.conf </ifmodule>
We need to now actually apply these rules to our site that we have just created. In our site.conf, we should add the following.
Include conf/rules/modsecurity_crs_10_config.conf
Include conf/rules/base_rules/*.conf
Include conf/rules/optional_rules/*.conf
When you startup your httpd now, you may get some "syntax" errors inside the mod_security configuration files. I just go in an delete/comment out the line in question. This is obviously okay if you are not using the technology the line pertains to. For example, you can probably safely comment out any php rules if you are not using php in your website. One in particular is about a data file missing for comment spam. You can go and get it (say google) or you can just deal with the line that is causing the issue.
After you startup apache, you will notice that the modsec_debug.log is getting populated. Feel free to test out mod_security. You can either use a tool like w3af, or just try putting sql injection as a parameter. You should see mod_security block it.
Sunday, January 9, 2011
mod_security, apache httpd, glassfish - Part 3
Part three.
Just to recap, we have now installed Glassfish, Java, all the deps for apache httpd, and the httpd itself. At this point you can startup apache httpd, and then try to navigate to the main page. You should get a 403 Forbidden. This is because we have not configured apache yet.
The apache httpd configuration file (located in apache_httpd_root/conf/httpd.conf) is a very well documented file. I'm not going to go through all of the details in the file, but I will touch on a few. Ideally, you will make a copy of the httpd.conf and start with a fresh file. From a security perspective, you want to have the configuration file to contain as little "junk" as possible. This will assist when you are trying to figure out a problem with your server configuration. It is just easier to read without all the explanations, plain and simple.
So basically, the first section is just getting general information. Don't worry about the ServerTokens Full just yet, we are going to use mod security to change our server signature in order to try and fool the script kiddies. The next section sets up the user you want apache to run as. If you want apache to run on port 80, we need to start apache as root. The user/group defined in the file is what apache will switch to after it binds to port 80. The load_modules section is simply loading all of our shared modules that we need. The Directory tag is setting up the base permissions. Basically, deny everyone for right now, until we get our site configured. As mentioned before, apache has wicked documentation, so I suggest you read it. Lets move on.
Our goal here is to get apache httpd talking to Glassfish. Once we establish that, we can work on implementing the advanced features of mod_security.
There are two ways to get apache httpd to play nice with Glassfish. One is to use mod_proxy. The other is to use mod_jk. Here is a very short list of pros and cons.
Mod_Proxy
pro: Super simple to setup.
con: Glassfish will see every request as originating from the apache httpd installation as opposed to someone out in the net. This is bad if you want glassfish to handle some type of security based on ip addresses.
Mod_JK
pro: More information provided to glassfish (ip address of originator etc.) Works on a specific language to connect which is faster than mod_proxy. Has built in support for workers to help support load.
cons: Hard to setup
For the purpose of this article, we are just going to use mod_proxy. I may come back later an make another post on using mod_jk.
We need to create a site.conf file with the following information.
Just to recap, we have now installed Glassfish, Java, all the deps for apache httpd, and the httpd itself. At this point you can startup apache httpd, and then try to navigate to the main page. You should get a 403 Forbidden. This is because we have not configured apache yet.
The apache httpd configuration file (located in apache_httpd_root/conf/httpd.conf) is a very well documented file. I'm not going to go through all of the details in the file, but I will touch on a few. Ideally, you will make a copy of the httpd.conf and start with a fresh file. From a security perspective, you want to have the configuration file to contain as little "junk" as possible. This will assist when you are trying to figure out a problem with your server configuration. It is just easier to read without all the explanations, plain and simple.
## Sample Apache httpd.conf configuration
ServerRoot "/path/to/apache/httpd"
ServerAdmin "your.address@you.com"
ServerTokens Full
ServerName myserver.myserveraddress.com
Listen 80
User apache
Group apache
LoadModule expires_module modules/mod_expires.so
LoadModule headers_module modules/mod_headers.so
LoadModule unique_id_module modules/mod_unique_id.so
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_connect_module modules/mod_proxy_connect.so
LoadModule proxy_ftp_module modules/mod_proxy_ftp.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule proxy_scgi_module modules/mod_proxy_scgi.so
LoadModule proxy_ajp_module modules/mod_proxy_ajp.so
LoadModule proxy_balancer_module modules/mod_proxy_balancer.so
LoadModule ssl_module modules/mod_ssl.so
LoadModule rewrite_module modules/mod_rewrite.so
DocumentRoot "/path/to/httpd/htdocs"
<Directory />
Options FollowSymLinks
AllowOverride None
Order deny,allow
Deny from all
</Directory>
ErrorLog "|/.../httpd/bin/rotatelogs -f /.../httpd/logs/error_log.%Y%m%d 86400"
TransferLog "|/.../httpd/bin/rotatelogs -f /.../httpd/logs/access_log.%Y%m%d 86400"
include conf/site.conf
So basically, the first section is just getting general information. Don't worry about the ServerTokens Full just yet, we are going to use mod security to change our server signature in order to try and fool the script kiddies. The next section sets up the user you want apache to run as. If you want apache to run on port 80, we need to start apache as root. The user/group defined in the file is what apache will switch to after it binds to port 80. The load_modules section is simply loading all of our shared modules that we need. The Directory tag is setting up the base permissions. Basically, deny everyone for right now, until we get our site configured. As mentioned before, apache has wicked documentation, so I suggest you read it. Lets move on.
Our goal here is to get apache httpd talking to Glassfish. Once we establish that, we can work on implementing the advanced features of mod_security.
There are two ways to get apache httpd to play nice with Glassfish. One is to use mod_proxy. The other is to use mod_jk. Here is a very short list of pros and cons.
Mod_Proxy
pro: Super simple to setup.
con: Glassfish will see every request as originating from the apache httpd installation as opposed to someone out in the net. This is bad if you want glassfish to handle some type of security based on ip addresses.
Mod_JK
pro: More information provided to glassfish (ip address of originator etc.) Works on a specific language to connect which is faster than mod_proxy. Has built in support for workers to help support load.
cons: Hard to setup
For the purpose of this article, we are just going to use mod_proxy. I may come back later an make another post on using mod_jk.
We need to create a site.conf file with the following information.
<VirtualHost *:80> ProxyPass / http://localhost:8080 ProxyPassReverse / http://localhost:8080 </VirtualHost>Start up your Glassfish (and confirm that it is running). Then start up your apache httpd. You should be able to go to http://localhost and get served the glassfish page.
mod_security, apache httpd, glassfish - Part 2
Part deux!
3) Compile and install pcre
5) Compile and install apache httpd
Now that we have installed all the dependencies, we need to configure and install the httpd. There are a couple of security issues that come up during the installation phase. Most admins would say (and I'm sure there is a "principle" written about it somewhere) that you should only compile what you actually need. Apache httpd comes with several built in modules that you would never ever use in a reverse proxy situation. You could approach this two ways.
1) Build all the modules with the --enable-shared=all options. This will build all modules as shared. Then you can simply edit your config as to which modules you want to load or not.
2) Use the configure script to only compile the modules you need
There are pros and cons to both approaches. With (1), all the modules are still there. If an attacker somehow got access to the conf directory, they could enable modules that could cause futher holes. More realistically, however, someone could accidently turn on a module, and that module could have a vulnerability. Option two allows you to have complete control on what is actually on the system. If you ever needed an additional module, however, you would have to recompile to build it. Because I'm game for pain, I'm going to go with option 2.
Just a recap, part 1 took us through installing glassfish. We now have glassfish up and running on our system.
This part will focus on setting up the apache httpd.
1) Compile and install apr-1.4.2
tar xzf apr-1.4.2.tar.gz cd apr-utils-1.3.10 CC="gcc -m64" ./configure --prefix=/path/to/deps --with-apr=/path/to/deps/bin/apr-1-config make; make test make install
The above code is going to be followed a lot (to a certain degree) so I'm only going to explain it this once. Untar the archive. Run the config script based on the parameters you want. The CC is compiler arguments and makes it compile for 64 bit (which is what my machine is). Make the binary, test it, install it.
You will also want to update your bashrc again.
LD_LIBRARY_PATH=<path/to/deps/lib>
Don't forget to source.
2) Compile and install apr-utils
tar xzf apr-util-1.3.10.tar.gz cd apr-utils-1.3.10 CC="gcc -m64" ./configure --prefix=/path/to/deps --with-apr=/paht/to/deps/bin/apr-1-config make; make test make install
3) Compile and install pcre
CC="gcc -m64" ./configure --prefix=/path/to/deps --enabled-shared
5) Compile and install apache httpd
Now that we have installed all the dependencies, we need to configure and install the httpd. There are a couple of security issues that come up during the installation phase. Most admins would say (and I'm sure there is a "principle" written about it somewhere) that you should only compile what you actually need. Apache httpd comes with several built in modules that you would never ever use in a reverse proxy situation. You could approach this two ways.
1) Build all the modules with the --enable-shared=all options. This will build all modules as shared. Then you can simply edit your config as to which modules you want to load or not.
2) Use the configure script to only compile the modules you need
There are pros and cons to both approaches. With (1), all the modules are still there. If an attacker somehow got access to the conf directory, they could enable modules that could cause futher holes. More realistically, however, someone could accidently turn on a module, and that module could have a vulnerability. Option two allows you to have complete control on what is actually on the system. If you ever needed an additional module, however, you would have to recompile to build it. Because I'm game for pain, I'm going to go with option 2.
C="gcc -m64" ./configure --disable-autoindex --disable-auth-basic --disable-cgi --disable-cgid --disable-userdir --enable-expires=shared --enable-headers=shared --enable-proxy=shared --enable-proxy-http=shared --enable-rewrite=shared --enable-so --enable-ssl=shared --enable-unique-id=shared --with-apr=/path/to/deps/bin/apr-1-config --with-apr-util=/path/to/deps/bin/apu-1-config --with-pcre=/path/to/deps/bin/pcre-config --with-ssl=/path/to/deps --prefix=/path/to/root/httpd-2.2.17
You can go through the configure options on your own if you wish. The only real requirements are ssl, rewrite, proxy, proxy-http, and unique-id.
After you have that all installed, you will want to run apache to ensure that it starts up. Because apache is currently configured for port 80, you will have to run it as root. Ideally you would get a startup script and sudo access to run it, but in this case, just su to root and run apachectl from the bin directory. You should get a Forbidden error message when you type localhost into your browser.
In the next part, I will talk about configuring apache httpd and using mod_proxy to connect to glassfish.
Subscribe to:
Posts (Atom)