Va una spada!

Bash: using fifo

February 21st, 2012

I wrote two bash scripts with consumer-producer paradigm on a fifo (named pipe).

This is a simple example:

## producer.sh
...
fifo="/path/to/fifo"

consumer.sh &

while [ ! -p "${fifo}" ]; do
sleep 1
done

while [...]; do
echo "message for logging"
echo "message for fifo" >"${fifo}"
done

...

## consumer.sh
...
fifo="/path/to/fifo"

trap "rm -f ${fifo}" INT EXIT TERM

[ -p "${fifo}" ] || mkfifo "${fifo}"

while read line <"${fifo}"; do
echo ${line}
done

...

These scripts have two problems: (1) for each echo (write) in the producer, bash open the fifo, write on it and close it (but this is not a real problem…); (2) for each read in the consumer (in the while loop), bash has the same behaviour (open, read, close). This causes a failure on the write in the producer (errno==EPIPE) when the fifo is closed for reading.

Solution: use the standard template open -> read/write (in the while loop) -> close

## producer.sh
...
fifo="/path/to/fifo"

consumer.sh &

while [ ! -p "${fifo}" ]; do
sleep 1
done

exec 10>"${fifo}"

while [...]; do
echo "message for logging"
echo "message for fifo" >&10
done

exec 10>&-
...

## consumer.sh
...
fifo="/path/to/fifo"

trap "rm -f ${fifo}" INT EXIT TERM

[ -p "${fifo}" ] || mkfifo "${fifo}"

exec 10<"${fifo}"

while read line <&10; do
echo ${line}
done

exec 10<&-
...

Scritto da: ffiore
Tags: bash
Posted in Sistemistica, Sviluppo | No Comments »

Add a plus to sqlplus

February 16th, 2012

Oracle default cli, sqlplus, doesn’t support readline, so you can’t navigate thru commands or search in history.
You’ll enable – between the others:

the four arrows
^R
ESC .
CTRL+U and CTRL+Y

Use one of those two readline wrapper you can really speed up your job:
* rlfe
* rlwrap (apt-get install rlwrap)

After building, run
# rlwrap sqlplus

Enjoy!

Scritto da: rpolli
Tags: linux, oracle, sqlplus
Posted in Sistemistica, Suggerimenti | No Comments »

Annotate C with GCC Function Attributes

February 13th, 2012

A nice feature of GCC – which is not standard – is support for annotate C code and selectively avoid or raise warnings.

Example 1. A function which has a willingly unused variable can avoid a warning at compile time

int my_function(int a, int b, __attribute__((unused)) int c_only_for_debug) {

printf(”print %d\n”, a+b);

#ifdef DEBUG

printf(”on debug print %d too\n”, c_only_for_debug);

#endif

}

More examples soon

Scritto da: rpolli
Tags: c, gcc, linux, programming
Posted in Suggerimenti, Sviluppo | No Comments »

Min(GW) is Most

February 1st, 2012

I had to monitor some machines from a windows environment. As I love to script anything I decided to play the game with bash…so my battleplan was:

– install a small *nix environment on my Windows vm;
– set a nice, resizable terminal window (putty sucks);
– write my script.

Since I’m an old experienced Cygwin user, I decided to use a lighter tool. Minimalistic Gnu for Windows was a nice pick. Now I love it! Install it from here http://sourceforge.net/projects/mingw/files/Installer/mingw-get-inst/

After you set it up – it’s easy – open the mingw terminal window, that still uses the #cmd console window, and you’ll get a nice bash shell!

As #cmd window is really ugly, you can install the MinGW Terminal issuing

# mingw-get install mintty

Then you just have to change the MinGW link on your desktop, telling it to use the mintty terminal”–mintty”

c:\mingw\msys\1.0\msys.bat –mintty

Now you have a working gnu enviroment. You can install new software with #mingw-get and forget putty, using our old beloved ssh

Scritto da: rpolli
Tags: bash, cygwin, gnu, mingw, windows
Posted in Sistemistica, Suggerimenti | No Comments »

Oracle Cheatsheet

January 10th, 2012

As I don’t usually work on Oracle…

Misc

* limit resultset to 10 rows
> SELECT … WHERE rownum<10

* dump output to file
> SPOOL /tmp/file.sql
> SELECT * from my_table;
> SPOOL OFF

Tables

* show tables
> SELECT table_name FROM user_tables;

* list table informations
> SELECT table_name, compression, compress_for FROM user_tables where table_name like ‘%MYSUFFIX’;

* describe tables;
> DESCRIBE $table; — not a valid sql statement, usable only by sqlplus

* show free space in tablespace
> SELECT file_name, tablespace_name, ROUND(MAXBYTES/1024000) FB, ROUND(bytes/1024000) MB from dba_data_files

* show lines in partitions
> SELECT partition_name, tablespace_name, compression, num_rows FROM dba_tab_partitions WHERE num_rows > 0 and table_name LIKE ‘%MYSUFFIX’;

Procedures

* list procedures
> SELECT * FROM all_procedures;
* list user procedures
> SELECT * FROM user_procedures;
* list names
> SELECT DISTINCT name FROM user_source WHERE type=’PROCEDURE‘;
* list content
> SELECT text FROM user_source WHERE type = ‘PROCEDURE‘ AND name=’ProcName’

* list a stored procedure code without headers

> set serveroutput ON
> set feedback off
> set linesize 200
> set ver off
> set pagesize 5000
> set heading off

Partitions & Tablespaces

* list tables and tablespaces
> SELECT TABLE_NAME,TABLESPACE_NAME FROM user_tables WHERE table_name LIKE ‘MY_PRJ_%’;

* list partitions and tablespaces
> SELECT PARTITION_NAME,TABLESPACE_NAME FROM dba_tab_partitions WHERE table_name LIKE ‘MY_PRJ_%’;

* drop partition
> alter table MY_TABLE drop partition MY_TABLE_201202161605

Formatting

A column with 20 char
>COL my_text_column FOR a20

A column with 3 digits +2decimal (9 is just a placeholder)
> COL my_number_column FOR 999.99

Scritto da: rpolli
Tags: cheatsheet, db, oracle
Posted in Sistemistica | 1 Comment »

Reentrant not safe (aka don’t syslog() on fork())

January 4th, 2012

Today I experience a deadlock using localtime_r() after forking a multi-thread program. I was quite surprised, because – well – using localtime_r() I thought to be in some way safe. But I was not

Shortly I use a logging function which calls localtime_r(): my_log(”message”);

And the following happens even if you use syslog(), because syslog() on most system uses localtime() [yes, the non reentrant one too].

After the child hung, I found that it was in futex_wait state using:
# ps -elf ;
Looking for a mutex I discovered that – as you (don’t) know – localtime_r() calls tzset(). It’s a libc function using a mutex (call it _tz_mutex) while mangling the current timezone.

The “bad” code was basically doing the following:
main() { ... spawn many threads using my_log() ... if (fork()==0) { my_log("I am the child"); execv("/bin/bash", ...); } }
This is bad, because could happen the following:

one of the parent threads runs my_log(), locking the _tz_mutex (which is global and not thread local) ;
before the _tz_mutex is released, the main thread forks;
fork() preserves the locked mutex, because it’s a global one, but closes the thread that locked it: causing the deadlock;
child runs my_log(), trying to lock _tz_mutex and hanging.

This behavior is described in the rationale of pthread_atfork():

When fork() is called, only the calling thread is duplicated in the child process. Synchronization variables remain in the same state in the child as they were in the parent at the time fork() was called. Thus, for example, mutex locks may be held by threads that no longer exist in the child process, and any associated states may be inconsistent.

Moral:
1. don’t trust functions just because they end with “_r”;
2. run execv() ASAP after fork(), as stated in man pthread_atfork():

It is suggested that programs that use fork() call an exec function very soon afterwards in the child process, thus resetting all states. In the meantime, only a short list of async-signal-safe library routines are promised to be available.

3. between fork() and execv() use only simple functions: printf(), dup(),… you can find a list of async-signal-safe functions in
#man 7 signal

Scritto da: rpolli
Tags: c, fork, linux, mutex, posix, thread, unix
Posted in Sviluppo | No Comments »

Looking inside dojo…and patching

November 25th, 2011

While developing a simple rest service I needed a simple web interface.

While a full-featured dojo is quite heavy, if you use dojo mobile without importing any dijit stuff you can speed up things a lot. And if you use an http cache for static contents thing will even get better!

I choose dojo mobile 1.7 because it has a nice Grid+Store implementation.

Grid+Store is a toy that renders the json output of an url into an html table or list. When the url response changes, the page is updated.

Moreover the Grid/List can bind each row to an action, so that if you click on a item, you’ll be directed to another page. In my case the main view shows a list of file, and clicking on one file you will see it’s attributes (size & co).

So, when clicking on a file I needed to issue an ajax request retrieving file info from the server.
Dojo implements this using the “callback” tag.
ex.


file.txt

The previous html is generated by a json string like the following, so with dojo Store you can dynamically build a list with actions connected to each item.
{ 'item' : { 'label' : 'file.txt', 'moveTo' : 'fileView', 'callback' : 'function() {renderfile("/path/to/file.txt");}' } }
In dojo 1.7 the callback stuff is broken, so after looking on dojo trac, I found and apply a patch. Still no way, but at least I got an error – while before the patch the callback parameter was simply ignorated.

The error was something like
Object "renderfile('/path/to/file.txt')" has no method apply

The apply() method is a js method that a function must have. Better: if a js object hasn’t the apply() method, then it’s not a function!

Using js console I found that the callback argument was interpreted as a string, and not as a function.

So I start investingating on the json I used to populate the grid. And I noted that the issue was related to the quote sign around the callback function.

In fact json encloses in string only strings, not numbers or functions. But the json was generated by the server, using python simplejson library. For that lib, a js function is a string, so it’s encosed in quote.

The solution was to find the place where the function was called in dojo, and add a test: if the object supposed to contain a method is a string, then you evaluate the string to a method.
>>>lang.js +if (typeof(scope[method].apply) == "undefined" ) { + scope[method] = eval(scope[method]); +} return scope[method].apply(scope, arguments || []);


>>>ViewController.js

+if (src.callback && typeof(src.callback.apply) == "undefined") {

+      src.callback = eval(src.callback);

+}

w.performTransition(moveTo, evt.detail.transitionDir, evt.detail.transition, src.callback && src, src.callback);

Scritto da: rpolli
Tags: dojo, javascript, json, mobile
Posted in Sviluppo | No Comments »

Antialiasing su Firefox

October 28th, 2011

Ecco dei semplici passi per abilitre l’antialias in Mozilla Firefox. Senza di esso i caratteri sono visualizzati con una scalettatura che ne rende difficile la lettura in particolar modo per quelli di dimensioni piccole. Attivando l’antialias dell’interfaccia grafica di Linux non serve per migliorare il rendering delle pagine web di Firefox.

Per fare questo bisogna:

Digitare about:config nella barra degl’indirizzi di firefox.
Cercare la voce gfx.use_text_smoothing_setting
Impostare la voce a “true”

Dopo il riavvio di firefox le pagine web saranno molto più belle e di facile lettura.

Buona navigazione!

Scritto da: dabbasciano
Posted in Uncategorized | No Comments »

SSL Certificate, Chromed

October 16th, 2011

It’s not a long time that Chrom* browsers added a certificate management panel. Not everybody knows – in fact – that they rely on the Network Security Service tools. To add an untrusted certificate to Chrom* without using the inteface you have to add it there.
To manage NSS you need to install certutil, contained in libnss

# apt-get install libnss3-tools

The NSS certificate database is stored in your home directory:

# ls /home/rpolli/.pki/nssdb -la
total 40
drwx------ 2 rpolli rpolli  4096 2011-10-16 19:24 .
drwx------ 4 rpolli rpolli  4096 2011-10-16 19:15 ..
-rw------- 1 rpolli rpolli 10240 2011-10-16 19:24 cert9.db
-rw------- 1 rpolli rpolli 13312 2011-10-16 19:24 key4.db
-rw------- 1 rpolli rpolli   529 2011-10-16 19:18 pkcs11.txt

You can list contained certificates with

# certutil -d sql:$HOME/.pki/nssdb  -L

Certificate Nickname                                         Trust Attributes
SSL,S/MIME,JAR/XPI

Get your certificate from your site with

#openssl s_client -connect www.mysite.org:443

you can script it a bit

# echo quit | openssl s_client -connect www.mysite.org:443 |& sed -ne '/--BEGIN/,/--END/p' > /tmp/mycert.pem

Convert the cert with

# openssl x509 -in /tmp/mycert.pem -inform PEM -outform DER -out /tmp/mycert.der

And add new certificates (in DER format)

# certutil -d sql:$HOME/.pki/nssdb -A -t "C,," -n www.mysite.it  -i /tmp/mycert.der

Now restart chrome and enjoy!

Scritto da: rpolli
Tags: certificate, chrome, chromium, ssl
Posted in Uncategorized | No Comments »

Serving ACL on Samba

October 13th, 2011

I was playing a bit with samba, and I guess if I was able to serve files using access control list (aka ACL).

Posix ACL

While standard unix permissions allow one owner, one group and everybody – with some tweekings like directory sticky bit – new filesystems like ext3 and xfs gave us a bit control more.

They implement POSIX ACL. This is an old but widely used standard. To enable ACL we should firstly ask to the filesystem to set them up.

#mount /home -o remouont,acl,user_xattr

Then we can start playing: create a file and get its unix permission.
First of all let’s use umask to disable other user access to newly created files

# umask 077
# touch  /home/rpolli/sample_acl.txt
# ls -la  /home/rpolli/sample_acl.txt
 -rw------- 1 rpolli rpolli 0 2011-10-13 17:28 /home/rpolli/sample_acl.txt

Thanks to umask nobody but the owner can access this file.
Then we get its acl with #getfacl and check that everything matches!

# getfacl  /home/rpolli/sample_acl.txt
getfacl: Removing leading '/' from absolute path names
# file:  home/rpolli/sample_acl.txt
# owner: rpolli
# group: rpolli
user::rw-
group::---
other::---

Now let’s give write permission to this file to the caldavd user, which is not in the rpolli group

#setfacl -m u:caldavd:rw /home/rpolli/sample_acl.txt
#getfacl /home/rpolli/sample_acl.txt
getfacl: Removing leading '/' from absolute path names
# file: home/rpolli/sample_acl.txt
# owner: rpolli
# group: rpolli
user::rw-
user:caldavd:rw-
group::---
mask::rw-
other::---

So, to our common sense, file permissions are no more 600, as there’s somebody that can read it. Let’s look at the ls output

# ls -l /home/rpolli/sample_acl.txt
-rw-rw----+ 1 rpolli rpolli 0 2011-10-13 17:35 sample_acl.txt

There’s an indicator that somebody can read it, and a “+” flag at the end of unix permissions, stating that this file uses some more security mechanism.

You can exclaim now “Very impressive, Kowalski, but…can it fly?”.

rpolli# sudo su - caldavd
caldavd$ ls /home/rpolli/
ls: cannot open directory /home/rpolli/: Permission denied
caldavd$ ls /home/rpolli/sample_acl.txt -la
-rw-rw----+ 1 rpolli rpolli 0 2011-10-13 17:35 /home/rpolli/sample_acl.txt

and finally

caldavd$ echo pippo>/home/rpolli/sample_acl.txt
caldavd$ cat /home/rpolli/sample_acl.txt
pippo

Serving it with samba

To serve a directory with Samba 3 we just have to add the following stanza to the smb.conf

[share]
   comment = Ioggstream Samba share
   read only = no
   path = /home/share/
   guest ok = no
   nt acl support = yes

First of all we need to share a folder. Disabling guests is optional, but to change ACL you have to authenticate: so no guests this time!
The compulsory statement is to allow “nt acl”.

Once we restart samba, we can browse our folder using a Windows Vista. Strangely enough the WindowsXP file browser doesn’t detect ACL on my server.

So open your client and go to \\192.168.0.7\ (that’s my samba ip, use yours ) and insert your credential.
Right click on your folder and select “Security” (Protezione in Italiano) and..voilà! You will be able to see and edit your files .permission!

Scritto da: rpolli
Tags: cifs, samba, security, sysadmin, windows
Posted in Sistemistica | No Comments »