Showing posts with label scripts. Show all posts
Showing posts with label scripts. Show all posts

Monday, January 23, 2023

Python: reading a text file - character


   Reading a text file in python3 (csv or txt) there is a character that can be appreciated using "more" in terminal but in python3 the situation is more complicated.


  $ more epa.csv

<U+FEFF>the text


   Python3 reads the file well, it doesn't throw an error, but that invisible "character" remains in the variables, the texts, etc. and can cause some inconvenience.


   The solution is to read the file and specify the encoding, something as simple as:


with open(FILENAME, encoding='utf-8-sig') as file:

     for line in file:


Explanation (taken from:

The Unicode character U+FEFF is the byte order mark, or BOM, and is used to tell the difference between big- and little-endian UTF-16 encoding.

Good luck,

Tuesday, September 1, 2020

Tiny script: how to get the RRSIG DNS records with python3

 (I guess there are many other ways to do this, even more elegant)

import dns.resolver
domain =
request = dns.message.make_query(domain, dns.rdatatype.ANY)
response = dns.query.tcp(request,'')

for item in str(response).splitlines( ):
  if 'RRSIG' in item: print (item)

Monday, February 29, 2016

Read a BGP live stream from CAIDA

  Read a BGP live stream from CAIDA and insert them into a BGP session

What do we need
  bgpreader from the bgpstream core package provided by Caida obtained in github

  We will read the BGP live stream feed using bgpreader, then the standard output of it will be redirected to a pipe file (mkfifo) where a perl script called bgpsimple will be reading this file. This very same script will established the BGP session against a BGP speaker and announce the prefixes received in the stream.

LAB Topology
  The configuration was already tested in Cisco & Quagga
  The BGP Speaker (Cisco/Quagga) has the IPv4 address
  The BGP Simple Linux box has the IP

How does it works?
  bgpreader has the ability to write his output in the -m format used by libbgpdump (by RIPENCC), this is the very same format bgpsimple uses as stdin. That's why myroutes is a PIPE file (created with mkfifo).



First install general some packages:
apt-get install apt-file libsqlite3-dev libsqlite3 libmysqlclient-dev libmysqlclient
apt-get install libcurl-dev libcurl  autoconf git libssl-dev
apt-get install build-essential zlib1g-dev libbz2-dev
apt-get install libtool git
apt-get install zlib1g-dev

Also intall wandio
git clone


cd wandio
./configure && ./make && ./make install

to test wandio:

Download bgp reader tarball from:

#ldconfig (before testing)

#mkfifo myroutes

to test bgpreader:
./bgpreader -p caida-bmp -w 1453912260 -m
(wait some seconds and then you will see something)

# git clone

Finally run everything
In two separate terminals (or any other way you would like to do it):

./bgpreader -p caida-bmp -w 1453912260 -m > /usr/src/bgpsimple/myroutes
./ -myas 65000 -myip -peerip -peeras 65000 -p myroutes

One more time, what will happen behind this?
bgpreader will read an online feed from a project called caida-bmp with starting timestamp 1453912260 (Jan 27 2016, 16:31) in "-m" format, It means a libbgpdump format (see references). The stardard output of all this will be send to the file /usr/src/bgpsimple/myroutes which is a "pipe file". At the same time, will create an iBGP session againts peer (a bgp speaker such as Quagga or Cisco). will read myroutes files and send what it seems in this file thru the iBGP Session.

Important information
- The BGP Session won't be established until there is something in the file myroutes
- eBGP multi-hop session are allowed
- You have to wait short time (few seconds) until bgpreaders start to actually see something and starts to announce to the BGP peer

References / More information:
-Part of the work was based on:

- Caida BGP Stream:

- bgpreader info:

- RIPE NCC libbgpdump:

- Introduction of "Named Pipes" (pipe files in Linux):

Saturday, January 12, 2013

Error 1017 in Cacti. MySQL. Blank Graphics

4/7/2012 5:37:49 PM - CMDPHP: Poller [0] ERROR: SQL Cell Failed!, Error: '1017 ', SQL: "SELECT count (*) FROM polle
WHERE r_time poller_id = 0 AND end_time> '0000-00-00 00:00:00 '"

4/7/2012 5:38:34 PM - CMDPHP: Poller [0] ERROR: SQL Cell Failed!, Error: '1017 ', SQL: "SELECT count (*) FROM polle
r_time WHERE end_time = '0000-00-00 00:00:00 '"

If the file cacti.log you see the above error tt indicates a failure with poller_output mysql table.
In this respect, I have seem three different solutions, order from the less risky to the highest.

1) Repair mysql table with a php script that brings cacti

# php $PATH-TO-CACTI/cli/repair_database.php

2) Repair the table with mysql command:

# mysql> REPAIR TABLE poller_output;

If you do not know the user information for the mysql database, it can be found here:


3) Remove the table and do it again (this solution despite being aggressive works perfectly and you do not lose the historic Cacti information). Just enters the mysql CLI and copy/paste the following:

- Table structure for table `poller_output`
DROP TABLE IF EXISTS `poller_output`;
CREATE TABLE `poller_output` (
MEDIUMINT local_data_id `` (8) unsigned NOT NULL default '0 ',
Rrd_name `` varchar (19) NOT NULL default'',
`Time` datetime NOT NULL default '0000-00-00 00:00:00 ',
`Output` text NOT NULL,
PRIMARY KEY (`local_data_id`, `rrd_name`, `time`)
- Dumping data for table `poller_output`
Poller_output `LOCK TABLES` WRITE;
/ *! 40000 ALTER TABLE `poller_output` DISABLE KEYS * /;
/ *! 40000 ALTER TABLE `poller_output` ENABLE KEYS * /;

Ready!, Then you can wait for the poller runs 15 minutes and you will have something in your graphs. If you want to force the poller command is as follows:

# /usr/bin/php-q /var /www/miServer-cacti/poller.php  -force

I hope it will be useful,

Saturday, January 5, 2013

Disable / shutdown iptables on Linux

Sometimes it is necessary to "shutdown" or disable our Linux iptables, the procedure depends on the Linux distribution you're using.  
1) Procedure if you are using Redhat, Fedora, Mandriva / Mandrake or Centos, you just have to run the following:
# service iptables save
# service iptables stop
# chkconfig iptables off
# / etc /init.d/iptables stop

2) How to disable iptables on Debian or Ubuntu

a) Create a script called fw.stop with the following contents:

# /bin/sh
echo "Stopping firewall and Allowing everyone ..."
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT

b) Give execute permission to the script:

# chmod + x / root / fw.stop
# chmod 755 fw.stop
c) You can run the scritp with the following command:
# ./fw.stop

More info at: