Hack into a program

Published in

NSUACMSC

4 min readApr 21, 2018

Hacking means unauthorized intrusion into a system. Even though the word hacking has many other aspects to it, whenever we hear this word we think of someone stealing a social media account of someone else. And now, we will explore on how our innocent looking programs can be exploited very easily by a clever hacker. Our final goal is to write safer programs.

Often you have heard from your programming language teacher that C is a very dangerous language. Really, in the wrong hands it’s a backfiring weapon. C always leaves it to the Programmer to check for the integrity of data. This gives the programmers a certain level of freedom, but at the same time, it also increases the chances of leaving trails for the hackers.

Look at this portion of code:

char str[10];
gets(str);

You have declared a string str of size 10. Then you are taking the input for str. As you know, the function gets() takes input as long as you don’t press Enter. So what happens when you insert a string that has more than 10 characters? You overflow the str variable.

Let’s investigate further

Let’s write a program that takes two strings as input and then prints them on the screen.

Now, let’s run this program in our command line.

C:\Users\USER\Desktop\Hack>BufferOverflow
Enter str2: 1234567891234
str1 is: 91234
str2 is: 1234567891234

Did you notice what happened? The str1 variable was supposed to hold the value “abcdefg” but instead it is showing “91234”. This kind of errors happen because, often we think it is the source code that is executed, where what actually gets executed is the object code. Hence, we don’t care about how the program will be executed in low level.

What went wrong?

The compiler took 8 bytes of memory for str2 and then choose another 8 bytes next to the str2 to store str1. So, when we have entered “1234567891234” for str2, the program didn’t do any bound checking for us and overflowed to the str1 variable. Look at the image below to picture it better:

This is why, when we print str1 we get from 9 to the first null character ‘\0’. The same way printing str2 gives us from 1 to the first null character ‘\0’ it encounters.

Now as a clever hacker, can you think of any way you can use this kind of exploitation?

More Theory (You had it coming)

Every C program has a heap and a stack for storing data. The heap contains all the dynamic data, while the stack contains our static variable data declared inside a function. So, every time you declare a variable, it gets stored on top of that stack. That is why, we found our str1 and str2 variables close to each other.

Time to get our hands dirty

It’s time to get real. Let’s venture into how a clever hacker can exploit your programs by keeping Buffer Overflow in mind.

Suppose, you have made a Library management system as your CSE project. Of course, you have a part in your program where the admin/librarian can access only with a password. This is a oversimplified version of your program.

Take some time to go through this program.

You have a checkPassword() function which prompts for a password and then returns 1 if the password match with “iLoveAngelPiya” otherwise return 0.

(Normally you don’t hardcode the password like this, but we are hardcoding it because the librarian has requested us to do so and also it keeps the code simple here).

Let’s run this code:

C:\Users\USER\Desktop\Hack>LibraryApp
Admin Login
Enter your password: 12345Access Denied.

Ohho, bad luck. The password isn’t 12345, so we couldn’t access admin control. Let’s try again:

C:\Users\USER\Desktop\Hack>LibraryApp
Admin Login
Enter your password: iLoveAngelPiya*****************
 Access Granted.
*****************

That was easy, because we knew the password. What if we wanted to access it anyway, without knowing the password?

C:\Users\USER\Desktop\Hack>LibaryApp
Admin Login
Enter your password: AAAAAAAAAAAAAAAAAAAAAAAAAAAA*****************
 Access Granted.
*****************

Wait, what???

Isn’t the password “iLoveAngelPiya”?
Then, how come we could access the program by entering many A’s?

Remember the image showing the memory overflowing at the top? Now, look at the checkPassword() function. Your variable password is supposed to be holding no more than 16 characters. But here we have entered 28 A’s. So where were the other 12 A’s stored? Well, they overflowed into other nearby variables. Notice that, you have declared a flag variable isValid right after the string variable password and the compiler decided to keep space for isValid close to the password. The isValid variable was initially set to 0. But, we managed to overflow to it with an arbitrary numbers of A’s. This changed the value of isValid from 0 to some other value. Finally, the checkSum() function returned a non zero value to the if() condition in main(). The if(condition) is true whenever there is a non zero integer in its condition. and granted admin access to us.

This exploit is only the tip of the iceberg. To find other exploits, you may require a high level of creativity and dedication.

Wrapping Up

Hacking is a way of life. And you should hack only to make yours and others life easier and safer. For the same reason you also should know how malicious hackers think. This way you can write better programs.