An introduction to Spring Python and IoC/DI (Inversion of Control/Dependency Injection)
Continuing on the last post, we’ll now be extending the code with an IoC (Inversion of Control)-driven configuration. Spring Python offers an IoC container which may be used for injecting dependencies between the application’s objects. The concept is that objects are to focus on their job and usually should be blissfully unaware of where the other objects they operate upon come from. The dependencies are dynamically injected so the control of who uses what is sort of inverted, it’s the container who knows what and when to distribute to others in order for everything to play nicely together and that’s why language lawyers can’t decide whether to call it IoC (Inversion of Control) or DI (Dependency Injection) but let that not stop us from applying it in practice 
And of course it’s only a concept, it’s not something magical, it’s simply a decent addition to your toolbox which you may like to get familiar with and Spring Python with its gentle learning curve is just the right tool. Throughout the article, Spring Python 1.1 will be used.
Picking off from where we left last time, here’s a bare bones ‘uptime’ server and its client.
# An 'uptime' server # stdlib import logging import commands # Spring Python from springpython.remoting.pyro import PyroServiceExporter # Configure logging subsystem logging.basicConfig(level=logging.DEBUG) class Service(object): def get_uptime(self): logging.info("get_uptime invoked") return commands.getoutput("uptime") # Create a service .. service = Service() # .. export it through Pyro .. service_exporter = PyroServiceExporter() service_exporter.service = service service_exporter.service_name = "Service" service_exporter.service_host = "127.0.0.1" service_exporter.service_port = "16099" # .. and start the server. service_exporter.after_properties_set()
# Client # stdlib import logging # Spring Python from springpython.remoting.pyro import PyroProxyFactory # Configure logging subsystem logging.basicConfig(level=logging.DEBUG) # Point the client to a server .. service = PyroProxyFactory() service.service_url = "PYROLOC://127.0.0.1:16099/Service" # .. invoke the service .. uptime = service.get_uptime() # and print it on the console. logging.info("Current uptime: [%s]" % uptime)
A word on Spring Python’s IoC syntax, there’s a plenty of choices to pick from, with three major ones being PythonConfig, YamlConfig and XMLConfig. Being agile Python folks as we are, we chose to use YamlConfig and PythonConfig for configuring things but it should be noted that all other config modes would’ve allowed us to achieve the same.
As always, it’s all up to you to decide what makes more sense in given circumstances though my recommendation is to use YamlConfig or XMLConfig with a strong preference on the former if you work in a diverse Python/Java/.NET environment with people who already know Spring Framework and PythonConfig/YamlConfig if you’re free to choose whatever you like.
So here’s the server’s basic configuration written in YAML..
# server-config.yml objects: - object: service_name str: Service - object: service_host str: 127.0.0.1 - object: service_port str: 16099
.. and here’s some code to read if off a file we stored it in:
# ioc-yaml-sample.py # Spring Python from springpython.config import YamlConfig from springpython.context import ApplicationContext config = YamlConfig("./server-config.yml") container = ApplicationContext(config) service_name = container.get_object("service_name") print "Hey, the name is '%s'" % service_name
YamlConfig’s instance reads the file, parses it, creates an internal cache of definitions found and then the config may be fed to ApplicationContext which is the actual class used for fetching objects from a container. Why is the class called ApplicationContext instead of Container, you wonder? Well in fact the actual container is called springpython.container.ObjectContainer and ApplicationContext subclasses it in order to add some additional goodies such as the possibility of hooking into an object’s lifecycle.
Let’s now introduce server to the config and let’s create the main application’s module, boringly named ‘main.py’, so that in result we have following files ..
. |-- client.py |-- main.py |-- server-config.yml `-- server.py
.. with such a contents ..
# client.py (no changes here so far) # stdlib import logging # Spring Python from springpython.remoting.pyro import PyroProxyFactory # Configure logging subsystem logging.basicConfig(level=logging.DEBUG) # Point the client to a server .. service = PyroProxyFactory() service.service_url = "PYROLOC://127.0.0.1:16099/Service" # .. invoke the service .. uptime = service.get_uptime() # and print it on the console. logging.info("Current uptime: [%s]" % uptime)
# main.py # stdlib import logging # Spring Python from springpython.config import YamlConfig from springpython.context import ApplicationContext config = YamlConfig("./server-config.yml") container = ApplicationContext(config) # Fetch configuration from the container .. log_level = container.get_object("log_level") log_format = container.get_object("log_format") # .. and configure logging. logger = logging.getLogger("") logger.setLevel(logging.getLevelName(log_level)) handler = logging.StreamHandler() formatter = logging.Formatter(log_format) handler.setFormatter(formatter) logger.addHandler(handler)
# server-config.yml objects: - object: service_name str: Service - object: service_host str: 127.0.0.1 - object: service_port str: 16099 - object: service class: server.Service - object: exporter class: springpython.remoting.pyro.PyroServiceExporter properties: service: {ref: service} service_name: {ref: service_name} service_host: {ref: service_host} service_port: {ref: service_port} - object: log_level str: DEBUG - object: log_format str: "%(asctime)s - %(levelname)s - %(process)d:%(threadName)s - %(name)s:%(lineno)d - %(message)s"
# server.py # stdlib import logging import commands class Service(object): def __init__(self): self.logger = logging.getLogger("%s:%s" % (__name__, self.__class__.__name__)) def get_uptime(self): self.logger.info("get_uptime invoked") return commands.getoutput("uptime")
So what have we achieved? The actual business logic has been decoupled from its configuration so that it doesn’t know anything about Pyro because, after all, should it care about it? Its business is to return the current uptime and that’s what it’s good at. That it’s going to be exposed through Pyro is also not that important for its core functionality, it’s Pyro today but could be Zope tomorrow, who knows it. PyroServiceExporter also happily lives in a state of not really being aware of what service it is publishing outside. Changing either of those should not mean touching the source code even if it meant merely adding a couple of lines of code. And the other stuff, it’s just a simple configuration we had to externalize anyway so why not put it into same place for consistency? To keep things simpler, the client code hasn’t been changed yet though it will be later on when we’ll be working with PythonConfig.
When choosing what to place in the container there’s always a danger of going too far and letting it all look awkwardly and unpythonic or just plain ugly. As a rule of thumb, your application surely can be explained in terms of some kind of components which need to be wired up together and that’s where the division line could be initially drawn. Another point is standard configuration details such as host, port or the logging format above, it’s something that sysadmins love to customize so it should allow for an easy access (and that incidentally is another reason why XMLConfig is something you should think twice about before making use of as sysadmins usually have their, let’s call it, well-grounded opinion about working with XML). Remember how several years ago some applications were doing crazy things like ..
<query id="get_customer">
<table name="customer" pkey="cust_no">
<columns>
<column name="cust_name" />
<column name="address" />
<column name="phone_no" />
</columns>
</table>
<operator name="EQ" />
</query>
.. instead of ..
get_customer = "SELECT cust_name, address, phone_no FROM customer WHERE cust_no=?"
.. and they used to call it being “XML-ready” or “XML-aware”? Well, if you catch yourself on doing similar things with IoC then you know you’ve gone too far
If you happened to jump directly into running the code above instead of reading the text then you probably were wondering why the server had been started even though nowhere in the code we explicitly told it to. That’s because PyroServiceExporter is an InitializingObject – it’s a subclass of springpython.context.InitializingObject which means its after_properties_set method gets called right after all of its properties are resolved and set in place and it just happens that PyroServiceExporter starts a new server thread in its after_properties_set hook. If you scroll up to the beginning of this post you’ll recall that odd-looking service_exporter.after_properties_set() line that was advertized to ’start the server’, which it did - but if you’re using IoC, the container will call it on your behalf so that you don’t have to remember about it.
Other interesting classes which your objects may wish to subclass are springpython.context.ObjectPostProcessor and springpython.context.DisposableObject. Subclassing ObjectPostProcessor gives you two new methods to override – post_process_before_initialization which will be invoked before after_properties_set and post_process_after_initialization which will be called after after_properties_set will have been called. DisposableObject on the other hand lets you implement a method called destroy which will be called at the time when the container itself will be shutting down; to be precise, ApplicationContext registers its ’shutdown_hook’ method to the Python stdlib’s atexit module and the shutdown hook is responsible for calling each of the disposable objects’ ‘destroy’ method; ‘destroy’ is the right place for your components to clean up after themselves, e.g. it can be used to release any external resources held during an object’s lifetime.
It’s worth noting that ApplicationContext may be passed in a list of configurations and although it won’t be used in examples, each of them may be of different type and that’s what I usually end up with, bits that I feel are more or less of a static nature are kept in YamlConfig and PythonConfig stores those which sometimes need to do something more before they’re ready to be used.
Before focusing on adding security features there’s still one thing that badly needs be improved, it’s the client code which knows nothing about IoC and breaks the DRY principle by duplicating configuration. In real world situation this could be in some degree expected, as both the server and a client could’ve been developed or operated by different teams or organizations but here it looks feebly, though at the same time it provides a good pretext for focusing a bit more on PythonConfig Let’s rewrite it all into PythonConfig while keeping in mind the DRY rule then:
. |-- client-main.py |-- client_config.py |-- common_config.py |-- server-main.py |-- server.py `-- server_config.py
# client-main.py # stdlib import logging # Spring Python from springpython.context import ApplicationContext # Our application from client_config import ClientConfig client_config = ClientConfig() container = ApplicationContext(client_config) # Fetch configuration from the container .. log_level = container.get_object("log_level") log_format = container.get_object("log_format") # .. and configure logging. logging.basicConfig(level=logging.getLevelName(log_level), format=log_format) # .. fetch the service .. service = container.get_object("service") # .. invoke it .. uptime = service.get_uptime() # and print the output. logging.info("Current uptime: [%s]" % uptime)
# client_config.py # Spring Python from springpython.config import Object from springpython.remoting.pyro import PyroProxyFactory # Our application from common_config import CommonConfig class ClientConfig(CommonConfig): @Object def service(self): service = PyroProxyFactory() service.service_url = "PYROLOC://%s:%s/%s" % (self.service_host(), self.service_port(), self.service_name()) return service
# common_config.py # stdlib import logging # Spring Python from springpython.config import Object from springpython.config import PythonConfig class CommonConfig(PythonConfig): @Object def service_host(self): return "127.0.0.1" @Object def service_port(self): return "16099" @Object def service_name(self): return "Service" @Object def log_level(self): return "DEBUG" @Object def log_format(self): return "%(asctime)s - %(levelname)s - %(process)d:%(threadName)s - %(name)s:%(lineno)d - %(message)s"
# server-main.py # stdlib import logging # Spring Python from springpython.context import ApplicationContext # Our application from server_config import ServerConfig server_config = ServerConfig() container = ApplicationContext(server_config) # Fetch configuration from the container .. log_level = container.get_object("log_level") log_format = container.get_object("log_format") # .. and configure logging. logging.basicConfig(level=logging.getLevelName(log_level), format=log_format)
# server.py (no changes here) # stdlib import logging import commands class Service(object): def __init__(self): self.logger = logging.getLogger("%s:%s" % (__name__, self.__class__.__name__)) def get_uptime(self): self.logger.info("get_uptime invoked") return commands.getoutput("uptime")
# server_config.py # Spring Python from springpython.config import Object from springpython.remoting.pyro import PyroServiceExporter # Our application from server import Service from common_config import CommonConfig class ServerConfig(CommonConfig): @Object def service(self): return Service() @Object def exporter(self): exporter = PyroServiceExporter() exporter.service = self.service() exporter.service_name = self.service_name() exporter.service_host = self.service_host() exporter.service_port = self.service_port() return exporter
Methods decorated with an @Object decorator are turned into container-managed objects and by default they are stored in an internal cache of singleton objects as opposed to being fetched each time they are requested. Objects can also be lazily-initialized when they are first time needed by someone. Finally, they can be abstract and/or have parents. Let’s discuss the options through.
@Object is a short-hand form of @Object(scope.SINGLETON) and these two are equivalent.
# Spring Python from springpython.config import Object from springpython.config import PythonConfig from foo import SomeClass class MyConfig(PythonConfig): @Object def my_singleton(self): singleton = SomeClass() return singleton
# Spring Python from springpython.context import scope from springpython.config import Object, PythonConfig from foo import SomeClass class MyConfig(PythonConfig): @Object(scope.SINGLETON) def my_singleton(self): singleton = SomeClass() return singleton
The code below defines a regular object which won’t be stored in any cache, its value, and if fact the object returned, will be different each time the ‘random_password’ will be requested from a container.
# stdlib from uuid import uuid4 # Spring Python from springpython.context import scope from springpython.config import Object, PythonConfig class MyConfig(PythonConfig): @Object(scope.PROTOTYPE) def random_password(self): return uuid4().hex
If an object is marked as being lazily-initialized (which isn’t the default behaviour) it won’t be created until someone will try to reference it for the first time. This is in contrast to the default container’s action which is to eagerly create an object upon first seeing its definition in the config. In Python programming terms, it means that the first method below will be invoked as soon as the container will come across it whereas the second one won’t be.
# Spring Python from springpython.config import Object from springpython.config import PythonConfig from foo import SomeClass class MyConfig(PythonConfig): @Object def my_singleton(self): singleton = SomeClass() return singleton
# Spring Python from springpython.config import Object from springpython.config import PythonConfig from foo import SomeClass class MyConfig(PythonConfig): @Object(lazy_init=True) def my_singleton(self): singleton = SomeClass() return singleton
As in real life, parent-child relationships can be a bit more complex because an object can be both a parent and a child of yet another objects. Objects that shouldn’t be fetched directly from the container are quite obviously called abstract ones. If an object wishes to subclass another one, it needs to name the parent in its parent attribute. Note that credentials and crm_credentials objects below need to be made prototypes as they would be otherwise cached and crm_dev_credentials, crm_uat_credentials and ivr_credentials would be referencing the same Python objects.
# Spring Python from springpython.context import scope from springpython.config import Object from springpython.config import PythonConfig class Credentials(object): def __init__(self, app=None, user=None, password=None): self.app = app self.user = user self.password = password def __str__(self): return "%s %s %s" % (self.app, self.user, self.password) class ServerConfig(PythonConfig): @Object(scope.PROTOTYPE, abstract=True) def credentials(self): return Credentials("MyApp") @Object(scope.PROTOTYPE, parent="credentials") def crm_credentials(self, credentials=None): credentials.user = "MYUSER" return credentials @Object(parent="crm_credentials") def crm_dev_credentials(self, credentials=None): credentials.password = "MYPASSWORD1" return credentials @Object(parent="crm_credentials") def crm_uat_credentials(self, credentials=None): credentials.password = "MYPASSWORD2" return credentials @Object(parent="credentials") def ivr_credentials(self, credentials=None): credentials.user = "FOO" credentials.password = "BAR" return credentials
Armed with all that knowledge we can finally start configuring security stuff. The method returning an uptime shouldn’t of course be troubled with authenticating incoming requests, that should be a responsibility of some kind of a security provider and the server should have a dependency on it. We’ll add a new module and update a few.
. |-- client-main.py |-- client.py |-- client_config.py |-- common_config.py |-- security.py |-- server-main.py |-- server.py `-- server_config.py
Only new or modified modules are shown below:
# security.py class UsernamePasswordCredentials(object): def __init__(self, username, password): self.username = username self.password = password def __str__(self): return "<%s at %s, username=%s>" % (self.__class__.__name__, hex(id(self)), self.username) class UsernamePasswordSecurityProvider(object): def __init__(self, database=None): self.database = database def authenticate(self, credentials): if credentials.username in self.database: return self.database[credentials.username] == credentials.password
# server.py # stdlib import logging import commands class Forbidden(Exception): pass class Service(object): def __init__(self, security_provider=None): self.security_provider = security_provider self.logger = logging.getLogger("%s:%s" % (__name__, self.__class__.__name__)) def get_uptime(self, credentials): self.logger.info("get_uptime invoked") if self.security_provider.authenticate(credentials): self.logger.info("Letting the client in") return commands.getoutput("uptime") self.logger.info("Unknown client %s" % credentials) raise Forbidden("No chance mate!")
# server_config.py # Spring Python from springpython.config import Object from springpython.remoting.pyro import PyroServiceExporter # Our application from server import Service from common_config import CommonConfig from security import UsernamePasswordSecurityProvider class ServerConfig(CommonConfig): @Object def service(self): service = Service() service.security_provider = self.security_provider() return service @Object def exporter(self): exporter = PyroServiceExporter() exporter.service = self.service() exporter.service_name = self.service_name() exporter.service_host = self.service_host() exporter.service_port = self.service_port() return exporter @Object def security_provider(self): security_provider = UsernamePasswordSecurityProvider() security_provider.database = self.security_database() return security_provider @Object def security_database(self): return {"FOO": "BAR"}
# client-main.py # stdlib import logging # Spring Python from springpython.context import ApplicationContext # Our application from client_config import ClientConfig from security import UsernamePasswordCredentials client_config = ClientConfig() container = ApplicationContext(client_config) # Fetch configuration from the container .. log_level = container.get_object("log_level") log_format = container.get_object("log_format") # .. and configure logging. logging.basicConfig(level=logging.getLevelName(log_level), format=log_format) # .. fetch the service .. service = container.get_object("service") # .. invoke it .. credentials = UsernamePasswordCredentials("FOO", "BAR") uptime = service.get_uptime(credentials) # and print the output. logging.info("Current uptime: [%s]" % uptime)
It’s the security provider’s burden to authenticate clients and server only needs a simple yes/no decision before returning data or raising an exception. Setting it up in that fashion, the server won’t have to be touched at all should we wish to change providers and start authenticating clients in some other way, for instance, through SSL certificates. And I know you’ll start yawning when I tell you once again that I think it’s not a provider’s business how to get its hand on the usernames/passwords database either, it just needs to operate on it.
A nice side-effect of not importing dependencies directly is that it’s easier to test the components in isolation without resorting to patching (which still comes handy at times) – replacing real objects with mocks and stubs becomes only a matter of a container’s returning different objects, though of course they still need to coded to given interfaces. Given that PythonConfig is pure Python code, it’s laughably easy to use it in tests, you just need to subclass the real config and return mock objects instead of real ones. For instance, we don’t really need to start a new Pyro thread just to make sure server and security provider can play nicely together – Pyro is just the transport layer and has nothing to do with how other components co-operate.
. |-- client-main.py |-- client.py |-- client_config.py |-- common_config.py |-- security.py |-- server-main.py |-- server.py |-- server_config.py |-- server_test_config.py `-- test_all.py
(Only new code below)
# server_test_config.py # Spring Python from springpython.config import Object # Our application from server_config import ServerConfig class _DummyExporter(object): pass class TestServerConfig(ServerConfig): @Object def exporter(self): return _DummyExporter()
# test_all.py # stdlib import unittest # Spring Python from springpython.context import ApplicationContext # Our application from server_test_config import TestServerConfig from security import UsernamePasswordCredentials class ServerTestCase(unittest.TestCase): def setUp(self): self.container = ApplicationContext(TestServerConfig()) self.default_credentials = UsernamePasswordCredentials("FOO", "BAR") def tearDown(self): self.container.shutdown_hook() def test_uptime(self): service = self.container.get_object("service") self.assertTrue(service.get_uptime(self.default_credentials)) if __name__ == "__main__": unittest.main()
Well, that’s about all for now. I’ve only touched the tip of the iceberg and showed you toy-size examples though it is almost always the case that IoC starts to shine when the code gets bigger and bigger - it’s just that creating a full-fledged application seemed a bit too much for a blog post Is IoC a panacea? Of course it isn’t, when not used judiciously you can quickly go from one extreme to another and instead of having dependencies intertwined right in the code you can lose the sight of the bigger picture of how control flows through an application’s building blocks simply because everything will seem too magical and declarative. As always, don’t overengineer things and don’t overdo with IoC, especially if you choose to use XML.
There’s also a question of whether Python really needs IoC, isn’t that really useful only in less dynamic languages like Java. Well, I truly believe that IoC should not really, or maybe not only, be thought of as a way to circumvent their way of dealing with changes to the source code, namely, that it needs to be recompiled each and every time, which is something that sometimes takes ages. It’s rather a fine way of bringing in more order and letting the programmers focus on their job while at the same time letting them be less afraid of introducing changes which could otherwise have a negative impact on applications. And it really helps in testing!
In case you ever need help with Spring Python or just want to chat, here’s the mailing list, a forum, a LinkedIn group, an IRC #springpython channel on Freenode network and there’s also a Spring Python 1.1 book upcoming.
