Using Perl to bypass UserAgent blocks

Let’s say you need to download/see a web content but the server is blocking your request due to your UserAgent.
For instance, an attacker has forged an web page where the malicious content is only available for Firefox running on Windows.

We can use LWP::UserAgent or urllib.
I coded in Perl due to project requirement.

The following code list all URLs inside this “malicious” domain ;)

use LWP::UserAgent;
use HTTP::Response;
use Data::Dumper;
use HTML::TreeBuilder 5 -weak; # Ensure weak references in use
use strict;
use warnings;

my %ua_config = (
	agent => "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.1",
	timeout => 60
);

my $ua = LWP::UserAgent->new(%ua_config);
my $urladdr = "http://kaiux.com/";
my $out_file="/tmp/blocker.com";

my $response = $ua->get($urladdr);

open (FH, '>', $out_file) or die "Could not open";
# if ($respon.) {save file }
print FH $response->decoded_content if $response->is_success;
close(FH);

## Parsing File by href
my $tree = HTML::TreeBuilder->new;
$tree->parse_file($out_file);

for ( @{ $tree->extract_links("a", "href") } ) {
	my($link, $element, $attr, $tag) = @$_;
	print $link, "\n";
}

For additional UserAgent:
http://useragentstring.com/pages/Firefox/

How to build GrSecurity using Debian way?

Last year I have created a set of steps teaching how to build GrSecurity patch for the vanila Kernel following the Debian way.

Basically is to download both Kernel source and GrSecurity patch and apply it against the newest kernel tree.

patch -p1 < ../grsecurity-3.1-3.2.68-201503251805.patch

and run:

fakeroot make deb-pkg

Thus, for every new patch and kernel available, I needed to follow all those steps again. Since I don’t like to run the same command over and over again, I have created a simple shellscript  to automatically build GrSecurity patch for a given  Kernel version.

mydebiankernelgrsec.sh has the following options:

 -c     config file path (/boot/config-3.2.68)
 -k     kernel version to download (only numbers 3.2.69)
 -l     list all Kernel versions available for download
 -t     download testing patch according new GrSecurity Policy

By the time I write this post, the following Kernel versions are available:

$ ./mydebiankernelgrsec.sh -l
Available Kernel versions for download

     4.5-rc3
     4.4.1
     4.3.5
     4.1.17
     3.18.26
     3.14.60
     3.12.53
     3.10.96
     3.4.110
     3.2.76
     2.6.32.70
     next-20160212

It worth note that GrSecurity has changed its policy for patch download; we can only download the testing patch or stable (costumer only – paid support).

Based on GrSecurity page, Test Patch is available only for kernel version from 3.1 up to 4.3.5. Lets build for version 4.3.5

Install all dependencies.

$ sudo apt-get install libncurses5-dev
$ sudo apt-get install kernel-package
$ sudo apt-get install fakeroot build-essential devscripts

Since I am running Debian Jessie, I will use the default kernel config file:

bash ./mydebiankernelgrsec.sh -t -k 4.3.5 -c /boot/config-config-3.16.0-4-amd64

You should accept every change you think is needed for your new kernel and follow the final steps:

"You should now run  'make menuconfig' inside of linux-4.3.5 and select and save"
"Security Options -> GrSecurity -> Configuration Method - Automatic"
"run the following command to generate .deb kernel file"
"$ fakeroot make deb-pkg"

And you are good to go!

All *.deb are available one directory above your current path.

$ ls ../*.deb
../linux-firmware-image-4.3.5-grsec_4.3.5-grsec-1_amd64.deb
../linux-image-4.3.5-grsec_4.3.5-grsec-1_amd64.deb
../linux-libc-dev_4.3.5-grsec-1_amd64.deb
../linux-headers-4.3.5-grsec_4.3.5-grsec-1_amd64.deb
../linux-image-4.3.5-grsec-dbg_4.3.5-grsec-1_amd64.deb

#http://en.wikibooks.org/wiki/Grsecurity/Configuring_and_Installing_grsecurity
#http://kernel-handbook.alioth.debian.org/ch-common-tasks.html#s-common-building

Checking IP reputation on DNS Blacklists

Checking for the IP reputation is not a new idea, however it is useful when one wants to know that an IP is safe for communication (email, vpn, etc…). Basically, all DNS Blacklists operate the same way.

Given an IP address format (a.b.c.d), one should send the DNS request following this format  d.c.b.a.dns.blacklist.tld.

For instance, the IP address 185.130.5.207  and cbl.abuseat.org  list result in: 207.5.130.185.cbl.abuseat.org

Using Net::DNS::Simple:

use Net::DNS::Simple;
# original IP: 185.130.5.207
my $res = Net::DNS::Simple->new("207.5.130.185.cbl.abuseat.org", "A");

if ( ($res->get_rcode() eq "NOERROR") && ($res->get_ancount() >= 1) ) {
    foreach my $line ( $res->get_answer_section() ) { 
     #sometimes an IP is listed with 127.0.0.[1-5]
        if ( $line =~ /127/ ) {
            print "Found IP: " , $line , "\n";
            exit 0;
        }
    }
}

Another very easy solution is to use DIG:

dig +short 207.5.130.185.cbl.abuseat.org A

Making DNS queries using Perl really simple

That is it!

#!/usr/bin/perl
use Net::DNS::Simple;
my $res = Net::DNS::Simple->new("www.google.com", "MX");
$res->print_domain();

I have been working with DNS for while using Perl and Net::DNS module. However something always annoyed me was the fact that when I needed to write a new tool, I had to spend too much time writing all the things that Net::DNS needs.

Given that I wrote a very simple module that is 100% based on Net::DNS module. I called it Net::DNS::Simple, previously named as MYResolver. There are other modules similar to Net::DNS usage easier such as Net::DNS::Resolver and  Net::DNS:Dig*.

Install

I always use ‘local::lib’/perlbrew strategy because I don’t want to mix my production server with crazy modules.

You need the Net::DNS module. Follow all ‘cpan’ procedure by default, just press enter.

$ cpan install Net::DNS

Assuming that you have Perl modules into your ~/perl5  (PERL_LOCAL_LIB_ROOT:

Cloning Net::DNS::Simple (It is not at CPAN yet)

$ git clone https://bitbucket.org/kaiorafael/net-dns-simple.git

Change for whatever place your PERL_LOCAL_LIB_ROOT is pointing to:

$ cd net-dns-simple; perl Makefile.PL INSTALL_BASE=/home/kaiorafael/perl5

Make and install it

$ make ; make install

I have this output

kaiorafael $ make
cp lib/Net/DNS/Simple.pm blib/lib/Net/DNS/Simple.pm
Manifying 1 pod document
kaiorafael $ make install
Manifying 1 pod document
Installing /home/kaiorafael/perl5/lib/perl5/Net/DNS/Simple.pm
Installing /home/kaiorafael/perl5/man/man3/Net::DNS::Simple.3pm
Appending installation info to /home/kaiorafael/perl5/lib/perl5/x86_64-kfreebsd-gnu-thread-multi/perllocal.pod

Please make sure that you have

$ ls  $PERL5LIB/Net/DNS/Simple.pm

Tested on Debian Jessie amd64/ Debian Wheezy kFreeBSD

Current Methods

*A word about About Net::DNS::Dig. It has a very week Query ID number generator and susceptible to Kaminsky’s DNS Bug. I have tried to tell the maintainer without success.

my $ID	= time % 65536;
print $ID;
[kaiorafael $] perl f.pl 
53953
[kaiorafael $] perl f.pl 
53954

http://cpansearch.perl.org/src/MIKER/Net-DNS-Dig-0.12/Dig.pm

Compiling Synapse for Debian Jessie using Debian Way

** I am not a Debian Developer / I can not guarantee that the following build wont crash your pc **

I have no idea why there is a version of synapse for Debian Wheezy and Sid, but no for Jessie (Debian Stable) [1]

You can follow these steps to build by yourself ‘synapse’ or you can download  my version [2]**.

If you don’t have build-essential, install it:

$ sudo apt-get install build-essential devscripts debhelper fakeroot

Install some dependencies before download the source code. It worth note that you also should install vala compiler (valc) and many other dependencies.

$ sudo apt-get install libvala-0.26-dev libzeitgeist-dev libdbus-glib-1-dev libgtk2.0-dev libglib2.0-dev libgee-dev libjson-glib-dev libunique-dev libgtkhotkey-dev librest-dev libnotify-dev libgtk-3-dev libgee-0.8-dev libkeybinder-3.0-dev libzeitgeist-2.0-dev

Download synapse for Debian Sid and compile over Debian Jessie. I don’t wanna mix my Debian Stable repos with Debian Sid. I always compile by myself.

$ mkdir buildsynapse ; cd buildsynapse
$ dget http://http.debian.net/debian/pool/main/s/synapse/synapse_0.2.99.1-1.dsc

The output should be something like that

synapse_0.2.99.1-1.dsc:
Good signature found
validating synapse_0.2.99.1.orig.tar.xz
validating synapse_0.2.99.1-1.debian.tar.xz
All files validated successfully.
dpkg-source: info: extracting synapse in synapse-0.2.99.1
dpkg-source: info: unpacking synapse_0.2.99.1.orig.tar.xz
dpkg-source: info: unpacking synapse_0.2.99.1-1.debian.tar.xz
dpkg-source: info: applying 0001-silence-desktop-entry-lacks-keywords-entry.patch

$ cd synapse-0.2.99.1

You can check other dependencies in ‘debian/control’ file. Make sure that no library is missing:

$ sudo apt-get install  intltool  pkg-config valac  libzeitgeist-2.0-dev libdbus-glib-1-dev libgtk-3-dev  libgee-0.8-dev  libjson-glib-dev  libkeybinder-3.0-dev  libnotify-dev

Now, we can run DebBuilder without checking file signature

$ debuild -us -uc

If everything worked, you should have the following output:

….
dpkg-deb: building pakage ‘synapse’ em ‘../synapse_0.2.99.1-1_amd64.deb’.
dpkg-deb: building pakage ‘synapse-dbg’ em ‘../synapse-dbg_0.2.99.1-1_amd64.deb’.
dpkg-genchanges  >../synapse_0.2.99.1-1_amd64.changes

You can install. The .deb file is located in upper directory

$ cd ..
$ sudo dpkg -i synapse_0.2.99.1-1_amd64.deb

You may check if everything went to its place

$ dpkg -L synapse

/.
/usr
/usr/bin
/usr/bin/synapse
/usr/share
/usr/share/man
/usr/share/man/man1
/usr/share/man/man1/synapse.1.gz
/usr/share/icons
/usr/share/icons/hicolor
/usr/share/icons/hicolor/scalable
/usr/share/icons/hicolor/scalable/apps
/usr/share/icons/hicolor/scalable/apps/synapse.svg
….

You can test it running

$ synapse

You are good to go!

You can use the old good style: ./configure ; make && make install or ./configure ; make && checkinstall

1- https://packages.debian.org/sid/synapse

2- http://mundosubmundo.kaiux.com/download/synapse_0.2.99.1-1_amd64.deb

 

How to create a Virtual Screen using xrand on Debian Jessie FluxBox desk

This is a very short tutorial how to setup a virtual screen also known as second monitor (no twin monitor).

My laptop has three main outputs, eDP1, VGA1, and HDMI1. eDP1 is LCD, VGA1 and HDMI1 output are on the side of my laptop.

You can identify your Outputs using the following command

$ xrandr | grep connected
 eDP1 connected 1920x1080+0+0 (normal left inverted right x axis y axis) 344mm x 194mm
 VGA1 disconnected 1440x900+1920+0 (normal left inverted right x axis y axis) 0mm x 0mm
 HDMI1 disconnected (normal left inverted right x axis y axis)

You can setup the Video resolution, in my case I have chosen 1440×900 for VGA1

$ xrandr --output VGA1 --mode 1440x900

After that you must choose where the Virtual Screen will be setup, on the right (--right-of), left  (--left-of), above (--above), or bellow (--below) of your major output video (eDP1)

$ xrandr --output VGA1 --right-of eDP1

https://pkg-xorg.alioth.debian.org/howto/use-xrandr.html
https://wiki.archlinux.org/index.php/Xrandr

O dono do nome: Facebucki e Amazô

Essa semana deparei-me com a notícia que a super e hiper interessante rede social Facebook entraria com um processo contra outra rede social voltada ao público evangélico; o FaceGlória.

facebuki processandoAo ler essa notícia lembrei do processo que levamos – eu e meu sócio – da grande livraria Amazon.com. Poucas pessoas sabem disso, mas acho que já chegou a hora de comentar o que aconteceu com a minha primeira empresa de Internet. Continue reading O dono do nome: Facebucki e Amazô

How to use IP to ASN from Team-Cymru using Perl?

Team-Cymru have a incredible service called IP to ASN which one enters an IP address and receives the AS number info.

I have implemented a Perl module (teamasn) to retrieve such info  in a very simple way. ‘teamasn’ module depends on ‘Net::DNS::Dig’ to sent a DNS query to Team-Cymru’s service.  My module is a parser to retrieve the ‘ANSWER’ section. Yes, Perl can be simple :)

For example, to find the AS number from IP 216.90.108.2 using DIG, type in the terminal:

 $ dig +short 2.108.90.216.origin.asn.cymru.com TXT
 "23028 | 216.90.108.0/24 | US | arin | 1998-09-25"

How to use it in Perl:

teamasn::get_dns_answer('216.90.108.2');

‘teamasn’ module returns an array composed by the AS number ( 23028), Network Class (216.90.108.0/24), Country Code (US), Registry (arin), and Allocation Date (1998-09-25). During my tests I have seen some IPs which ‘Allocation Date’ is absent, therefore one needs to check the array size to know how much info ‘teamasn’ got.

Here it goes a sample code:

#!/usr/bin/perl
# @kaiux
use teamasn;
use strict;
use warnings;
use v5.14;

# IPV4 only :/
my @ip_list = qw(200.242.79.172 200.141.79.11 199.16.158.104 94.100.180.202 217.69.139.200 192.168.0.1);

foreach my $ip (@ip_list)
{
	my @response = teamasn::get_dns_answer($ip);
	if ( $response[0] == -1 )
	{
		say "There is no info for: ", $ip;
	}
	else
	{
		say "AS: ", $response[0];
		say "NC: ", $response[1];
		say "CC: ", $response[2];
		say "RI: ", $response[3];
		say "DT: ", $response[4];
	}
}

One can find the ‘teamasn’ source code at: https://bitbucket.org/kaiorafael/dns_tools/

I will not push ‘teamasn’  to CPAN because it is very premature code and I need extra time to add new features such as:

get_asn: retrieve the AS number only
get_nwc: retrieve the network class only
get…etc…
IPV6….

There are other modules to request AS info such as Net::Whois, however it is HTTP-based and for my needs DNS is better because it is very fast.

Minicurso sobre Botnets – SBSEG 2014.

Tenho investigado o funcionamento de botnets e métodos de detecção através da análise do tráfego DNS em meu doutoramento. Botnets são redes de computadores infectados que permitem ao atacante controle remoto.

Em Novembro de 2014 apresentei*  um minicurso sobre o tema no SBSEG 2014 – XIV Simpósio Brasileiro em Segurança da Informação e de Sistemas Computacionais. No material exposto os participantes puderam entender o Ciclo de Vida das Botnets, ataques frequentes, como os protocolos de rede são usados por operadores das Botnets, meios de propagação e abordagens de detecção.

Para o curso não ser 100% teórico, implementei um bot que usa o tráfego IRC e outro para o HTTP. O primeiro foi um programa em ‘python’ que utiliza ‘sockets’ e conecta a um servidor IRC. O segundo é um ShellScript que faz um parser dos Tweets de um usuário no Twitter. O objetivo era mostrar que as redes sociais podem ser utilizadas como Comando e Controle. Conheça o KaiuxBot.

O material do minicurso está disponível em: http://tinyurl.com/sbseg2014

Obrigado a todos que puderam participar.

*Kaio R. S. Barbosa (UFAM) (apresentador), Eduardo Souto (UFAM), Eduardo Feitosa (UFAM) e Gilbert Martins (UFAM).

Agradecimentos à Haline Oliveira pela ajuda no lab.