[future]

Visions and ideas for further development of the py lib

This document tries to describe directions and guiding ideas for the near-future development of the py lib. Note that all statements within this document - even if they sound factual - mostly just express thoughts and ideas. They not always refer to real code so read with some caution.

1   Distribute tests ad-hoc across multiple platforms

After some more refactoring and unification of the current testing and distribution support code we'd like to be able to run tests on multiple platforms simultanously and allow for interaction and introspection into the (remote) failures.

2   Make APIGEN useful for more projects

The new APIGEN tool offers rich information derived from running tests against an application: argument types and callsites, i.e. it shows the places where a particular API is used. In its first incarnation, there are still some specialties that likely prevent it from documenting APIs for other projects. We'd like to evolve to a py.apigen tool that can make use of information provided by a py.test run.

3   Consider APIGEN and pdb integration

The information provided by APIGEN can be used in many different ways. An example of this could be to write an extension to pdb which makes it available. Imagine you could issue a pdb command "info <function name>" and get information regarding incoming, and outgoing types, possible exceptions, field types and call sites.

4   Distribute channels/programs across networks

Apart from stabilizing setup/teardown procedures for py.execnet, we'd like to generalize its implementation to allow connecting two programs across multiple hosts, i.e. we'd like to arbitrarily send "channels" across the network. Likely this will be done by using the "pipe" model, i.e. that each channel is actually a pair of endpoints, both of which can be independently transported across the network. The programs who "own" these endpoints remain connected.

5   Benchmarking and persistent storage

For storing test results, but also benchmarking and other information, we need a solid way to store all kinds of information from test runs. We'd like to generate statistics or html-overview out of it, but also use such information to determine when a certain test broke, or when its performance decreased considerably.

6   Refactor path implementations to use a Filesystem Abstraction

It seems like a good idea to refactor all python implementations to use an internal Filesystem abstraction. The current code base would be transformed to have Filesystem implementations for e.g. local, subversion and subversion "working copy" filesystems. Today the according code is scattered through path-handling code.

On a related note, Armin Rigo has hacked pylufs and more recently has written pyfuse which allow to implement kernel-level linux filesystems with pure python. Now the idea is that the mentioned filesystem implementations would be directly usable for such linux-filesystem glue code.

In other words, implementing a memoryfs or a dictfs would give you two things for free: a filesystem mountable at kernel level as well as a uniform "path" object allowing you to access your filesystem in convenient ways. (At some point it might even become interesting to think about interfacing to reiserfs v4 features at the Filesystem level but that is a can of subsequent worms).

7   Integrate interactive completion

It'd be nice to integrate the bash-like rlcompleter2 python command line completer into the py lib, and making it work remotely and with pdb.

8   Consider more features

There are many more features and useful classes that might be nice to integrate. For example, we might put Armin's lazy list implementation into the py lib.