gefira blog

I’ve just pushed a new feature to the lp:~pymqi-dev/pymqi/string-filter branch that makes it possible to use WebSphere MQ PCF/MQAI filters with Python. That should be a nice addition if anyone’s thinking of creating cross-platform open-source equivalents of various Windows-only freeware support packs there are already available. Or just for anyone writing slightly more advanced code for administering MQ objects.

So how does it work?

You sure remember how to use PCFExecute, right? Well, the class has just been extended and now accepts a new parameter, a list of filters to use when querying MQ objects. The API has been more or less inspired by how Django’s QuerySets look like which basically means it’s a higher level one then what you’d be normally using with plain MQ verbs. See for yourself, the code below fetches all the queues that have a description starting with a ‘WebSphere MQ Selection*’ pattern and the current depth of 10 and greater:

import pymqi, CMQC, CMQCFC
 
queue_manager = 'QM01'
channel = 'SVRCONN.1'
host = '192.168.1.139'
port = '1434'
conn_info = '%s(%s)' % (host, port)
 
qmgr = pymqi.connect(queue_manager, channel, conn_info)
pcf = pymqi.PCFExecute(qmgr)
 
attrs = {CMQC.MQCA_Q_NAME :'*', CMQC.MQIA_Q_TYPE : CMQC.MQQT_LOCAL}
 
desc_filter = pymqi.Filter(CMQC.MQCA_Q_DESC).like('WebSphere MQ Selection*')
depth_filter = pymqi.Filter(CMQC.MQIA_CURRENT_Q_DEPTH).greater(10)
 
res = pcf.MQCMD_INQUIRE_Q(attrs, [desc_filter, depth_filter])
print(res)

It’s still a feature branch not trunk, and even if it were trunk it still hasn’t been released in PyMQI 1.3, and even then, there still would be PyMQI 1.4, so if you feel like anything should be changed or rather work differently, please speak up! Cheers!

@fourthrealm

I wonder if anyone is willing to somehow tackle the idea I’d like to introduce in this post. Or maybe just tell me it’s all crap, doesn’t make any sense and maybe even sounds fine but will sure get hairy when one gets into details.

So the deal is, among other sites, I happen to have a user account over at Launchpad and Ohloh too. Fair enough. Now the thing is, there’s this karma concept associated with having an account. Basically, the more work you do at the site – be it resolving bugs, dealing with user questions, committing to repos, etc. – the more karma points you receive. The points can’t be traded for anything, no I guess it’s just a more or less interesting way to reaffirm one’s importance in this cold world full of people making quick judgements based on brief and cursory acquaintances, if you’re into that sort of things.

Anyway, about the karma. The nicer citizen of the open source world you are, the more karma you have, quite simple. But the trouble is, when you go to a new site, you need to build it all up from the very bottom, you have no karma in that new place. You’re the same person but the site doesn’t know you so it doesn’t give a fig about your having collected oh-so-many points on other sites – which is of course just another trivial example of the mismatch that underlies the difference between proving that you are yourself and that the computer you used at a certain time was really operated by you. Now, I couldn’t care less about it all – and still I’m not sure what to think about the idea for that matter – but the issue that I’m really thinking about is that the new sites sometimes require your proving that you’re a sane person, usually in the sense of not being a spammer. For instance, there are some forums that require that you post at least one message without links before you can post something that contains links – which is a simple and probably efficient spam protection. So what I had to do the other day at one forum was to post a silly message in a thread that I wasn’t really prepared to take part in before I could post a release announcement of one of the projects I participate in. I’m sure you can see where I’m coming from, had the forum known that I wasn’t that much of a jerk they wouldn’t've asked me for anything, they would’ve just somehow fetched my karma information, would’ve nodded to it and would’ve let me post the message just like that. And I think it could work in a general case. Why should you require me to prove I’m not a moron if your good friends over at the other great open source hub have already come to the conclusion that I’m an OK person?

I’m not really that convinced it should be done but I can see at least that one point about posting links that could be made easier with the concept we’re talking here. It’s likely that it could be useful whenever there’s any sort of reward dependent on the user’s reputation involved, that scenarios are likely candidates. And I certainly don’t want to even think how to properly implement it, I haven’t even written a single karma-providing system like the ones mentioned previously so I have no clue what kind of a common data model we’re talking about and how useful a generalization of what particular sites use could be. Just throwing out some idea, nothing more.

@fourthrealm

One of the niceties of sec-wall, the security proxy, is that it’s written in Python which means not only that one doesn’t have to learn yet another obscure configuration format summoned from the deepest levels of R’lyeh but also that it’s very easy to extend the config.py file.

For instance, by default all of the secrets need to be given in clear text and that can probably be sub-optimal for some people, so why don’t we customize the config file and have it use GnuPG with symmetric decryption via the python-gnupg wrapper?

Let’s install python-gnupg first (piece of cake):

$ sudo pip install python-gnupg

Let’s prepare the secret.ini file, an INI-formatted configuration file which contains all the secrets:

$ cat secret.ini
[secret]
INSTANCE_SECRET=756817c1e2b8443c9d6f728da9b2b5fd
admin=MySecretPassword
$

The file will be now encrypted using GnuPG (let’s say the password is ’123′) ..

$ gpg --symmetric secret.ini
$

.. the result of which is a new secret.ini.gpg file and secret.ini may now be removed:

$ rm secret.ini
$

OK, we need the accompanying config.py file now, just like the one below:

# -*- coding: utf-8 -*-
 
# stdlib
import cStringIO, getpass, sys, uuid
from ConfigParser import ConfigParser
 
# python-gnupg
import gnupg
 
# Where are we expecting the secrets to be.
gpg_file = open('./secret.ini.gpg', "rb")
 
passphrase = getpass.getpass('Enter passphrase (will not be echoed back): ')
 
gpg = gnupg.GPG()
ini_data = str(gpg.decrypt_file(gpg_file, passphrase=passphrase))
 
# Exit early on an incorrect passphrase.
if not ini_data:
    print("Incorrect passphrase.")
    sys.exit(1)
 
# The passphrase was OK, proceed on to parsing the INI data.
config = ConfigParser()
config.readfp(cStringIO.StringIO(ini_data))
 
# Use the GPG-encrypted data.
INSTANCE_SECRET = config.get('secret', 'INSTANCE_SECRET')
 
# May be shared with the outside world.
INSTANCE_UNIQUE = uuid.uuid4().hex
 
# ##############################################################################
 
def admin():
    return {
        'basic-auth': True,
        'basic-auth-username':'user',
        'basic-auth-password':config.get('secret', 'admin'), # Uses GPG now.
        'basic-auth-realm':'Secure area',
        'host': 'http://example.com'
    }
 
def default():
    return {
        'ssl': True,
        'ssl-cert': True,
        'ssl-cert-commonName':INSTANCE_SECRET,
        'host': 'http://' + INSTANCE_SECRET
    }
 
urls = [
    ('/admin/', admin()),
    ('/*', default()),
]

Try starting the proxy now. If everything’s OK, if the passphrase you’ve supplied has been the same as the one used for encrypting the original INI file, the sec-wall instance will start in background as usual:

$ sec-wall --start .
Enter passphrase (will not be echoed back):
$ echo $?
0
$

If on the other hand the password wasn’t ’123′, the proxy won’t start:

$ sec-wall --start .
Enter passphrase (will not be echoed back):
Incorrect passphrase.
$ echo $?
1
$

And that’s it! Couldn’t have been simpler, eh?

@fourthrealm

This is mostly a copy of the announcement sent to the python-announce@ list, although I’ve added a couple more usage examples at the end of the post.

sec-wall is a feature packed high-performance security proxy which has many interesting features, including the support for SSL/TLS, WS-Security, HTTP Auth Basic/Digest, extensible authentication schemes based on custom HTTP headers and XPath expressions, powerful URL matching/rewriting and an optional headers enrichment.

sec-wall uses and is built on top of several fantastic Python open source technologies, such as gevent, Spring Python, Pesto, lxml, zdaemon or PyYAML and is meant to be highly customizable and easy to use. Good performance, tests, documentation and building an awesome community are at the very heart of the project.

Here are  examples showing how little is needed to secure a backend server with HTTP Basic Auth, SSL/TLS client certificates and WS-Security.

# -*- coding: utf-8 -*-
 
# stdlib
import uuid
 
# Don't share it with anyone.
INSTANCE_SECRET = '5bf4e78c256746eda2ce3e0e73f256d0'
 
# May be shared with the outside world.
INSTANCE_UNIQUE = uuid.uuid4().hex
 
def default():
    return {
        'basic-auth':True,
        'basic-auth-username':'MyUser',
        'basic-auth-password':'MySecret',
        'basic-auth-realm':'Secure area',
        'host': 'http://example.com'
    }
 
urls = [
    ('/*', default()),
]

# -*- coding: utf-8 -*-
 
# stdlib
import os.path as path, uuid
 
# Don't share it with anyone.
INSTANCE_SECRET = '5bf4e78c256746eda2ce3e0e73f256d0'
 
# May be shared with the outside world.
INSTANCE_UNIQUE = uuid.uuid4().hex
 
# Useful constants
cur_dir = path.dirname(__file__)
 
# Crypto
keyfile = path.join(cur_dir, './crypto/server-priv.pem')
certfile = path.join(cur_dir, './crypto/server-cert.pem')
ca_certs = path.join(cur_dir, './crypto/ca-cert.pem')
 
server_type = 'https'
 
def default():
    return {
        'ssl': True,
        'ssl-cert': True,
        'ssl-cert-commonName': 'My Client',
        'ssl-cert-organizationName': 'My Company',
        'host': 'http://example.com'
    }
 
urls = [
    ('/*', default()),
]

# -*- coding: utf-8 -*-
 
# stdlib
import uuid
 
# Don't share it with anyone.
INSTANCE_SECRET = '5bf4e78c256746eda2ce3e0e73f256d0'
 
# May be shared with the outside world.
INSTANCE_UNIQUE = uuid.uuid4().hex
 
def default():
    return {
        'wsse-pwd': True,
        'wsse-pwd-username': 'MyUser',
        'wsse-pwd-password': 'MySecret', # Needs to be given in clear text
        'wsse-pwd-reject-empty-nonce-creation': True,
        'wsse-pwd-reject-stale-tokens': True,
        'wsse-pwd-reject-expiry-limit': 120,
        'wsse-pwd-nonce-freshness-time': 120,
        'host': 'http://example.com'
    }
 
urls = [
    ('/*', default()),
]

Links:

Project’s homepage: http://sec-wall.gefira.pl/
Getting started: http://sec-wall.gefira.pl/documentation/getting-started/index.html
Usage examples: http://sec-wall.gefira.pl/documentation/usage-examples/index.html
Twitter: https://twitter.com/fourthrealm
Blog: http://www.gefira.pl/blog
IRC: #sec-wall channel on Freenode network

Cheers!

@fourthrealm