As usual, Python's standard library received a number of enhancements and
bug fixes. Here's a partial list of the most notable changes, sorted
alphabetically by module name. Consult the
Misc/NEWS file in the source tree for a more
complete list of changes, or look through the CVS logs for all the
details.
The array module now supports arrays of Unicode
characters using the "u" format character. Arrays also now
support using the += assignment operator to add another array's
contents, and the *= assignment operator to repeat an array.
(Contributed by Jason Orendorff.)
The bsddb module has been replaced by version 4.1.6
of the PyBSDDB package,
providing a more complete interface to the transactional features of
the BerkeleyDB library.
The old version of the module has been renamed to
bsddb185 and is no longer built automatically; you'll
have to edit Modules/Setup to enable it. Note that the new
bsddb package is intended to be compatible with the
old module, so be sure to file bugs if you discover any
incompatibilities. When upgrading to Python 2.3, if the new interpreter is compiled
with a new version of
the underlying BerkeleyDB library, you will almost certainly have to
convert your database files to the new version. You can do this
fairly easily with the new scripts db2pickle.py and
pickle2db.py which you will find in the distribution's
Tools/scripts directory. If you've already been using the PyBSDDB
package and importing it as bsddb3, you will have to change your
import statements to import it as bsddb.
The new bz2 module is an interface to the bz2 data
compression library. bz2-compressed data is usually smaller than
corresponding zlib-compressed data. (Contributed by Gustavo Niemeyer.)
A set of standard date/time types has been added in the new datetime
module. See the following section for more details.
The Distutils Extension class now supports
an extra constructor argument named depends for listing
additional source files that an extension depends on. This lets
Distutils recompile the module if any of the dependency files are
modified. For example, if sampmodule.c includes the header
file sample.h, you would create the Extension object like
this:
Modifying sample.h would then cause the module to be recompiled.
(Contributed by Jeremy Hylton.)
Other minor changes to Distutils:
it now checks for the CC, CFLAGS, CPP,
LDFLAGS, and CPPFLAGS environment variables, using
them to override the settings in Python's configuration (contributed
by Robert Weber).
Previously the doctest module would only search the
docstrings of public methods and functions for test cases, but it now
also examines private ones as well. The DocTestSuite(
function creates a unittest.TestSuite object from a set of
doctest tests.
The new gc.get_referents(object) function returns a
list of all the objects referenced by object.
The getopt module gained a new function,
gnu_getopt(), that supports the same arguments as the existing
getopt() function but uses GNU-style scanning mode.
The existing getopt() stops processing options as soon as a
non-option argument is encountered, but in GNU-style mode processing
continues, meaning that options and arguments can be mixed. For
example:
The gzip module can now handle files exceeding 2 Gb.
The new heapq module contains an implementation of a
heap queue algorithm. A heap is an array-like data structure that
keeps items in a partially sorted order such that, for every index
k, heap[k] <= heap[2*k+1] and
heap[k] <= heap[2*k+2]. This makes it quick to
remove the smallest item, and inserting a new item while maintaining
the heap property is O(lg n). (See
http://www.nist.gov/dads/HTML/priorityque.html for more
information about the priority queue data structure.)
The heapq module provides heappush() and
heappop() functions for adding and removing items while
maintaining the heap property on top of some other mutable Python
sequence type. Here's an example that uses a Python list:
The IDLE integrated development environment has been updated
using the code from the IDLEfork project
(http://idlefork.sf.net). The most notable feature is that the
code being developed is now executed in a subprocess, meaning that
there's no longer any need for manual reload() operations.
IDLE's core code has been incorporated into the standard library as the
idlelib package.
The imaplib module now supports IMAP over SSL.
(Contributed by Piers Lauder and Tino Lange.)
The itertools contains a number of useful functions for
use with iterators, inspired by various functions provided by the ML
and Haskell languages. For example,
itertools.ifilter(predicate, iterator) returns all elements in
the iterator for which the function predicate() returns
True, and itertools.repeat(obj, N) returns
objN times. There are a number of other functions in
the module; see the package's reference
documentation for details.
(Contributed by Raymond Hettinger.)
Two new functions in the math module,
degrees(rads) and radians(degs),
convert between radians and degrees. Other functions in the
math module such as math.sin() and
math.cos() have always required input values measured in
radians. Also, an optional base argument was added to
math.log() to make it easier to compute logarithms for
bases other than e and 10. (Contributed by Raymond
Hettinger.)
Several new POSIX functions (getpgid(), killpg(),
lchown(), loadavg(), major(), makedev(),
minor(), and mknod()) were added to the
posix module that underlies the os module.
(Contributed by Gustavo Niemeyer, Geert Jansen, and Denis S. Otkidach.)
In the os module, the *stat() family of
functions can now report fractions of a second in a timestamp. Such
time stamps are represented as floats, similar to
the value returned by time.time().
During testing, it was found that some applications will break if time
stamps are floats. For compatibility, when using the tuple interface
of the stat_result time stamps will be represented as integers.
When using named fields (a feature first introduced in Python 2.2),
time stamps are still represented as integers, unless
os.stat_float_times() is invoked to enable float return
values:
In Python 2.4, the default will change to always returning floats.
Application developers should enable this feature only if all their
libraries work properly when confronted with floating point time
stamps, or if they use the tuple API. If used, the feature should be
activated on an application level instead of trying to enable it on a
per-use basis.
The optparse module contains a new parser for command-line arguments
that can convert option values to a particular Python type
and will automatically generate a usage message. See the following section for
more details.
The old and never-documented linuxaudiodev module has
been deprecated, and a new version named ossaudiodev has been
added. The module was renamed because the OSS sound drivers can be
used on platforms other than Linux, and the interface has also been
tidied and brought up to date in various ways. (Contributed by Greg
Ward and Nicholas FitzRoy-Dale.)
The new platform module contains a number of functions
that try to determine various properties of the platform you're
running on. There are functions for getting the architecture, CPU
type, the Windows OS version, and even the Linux distribution version.
(Contributed by Marc-André Lemburg.)
The parser objects provided by the pyexpat module
can now optionally buffer character data, resulting in fewer calls to
your character data handler and therefore faster performance. Setting
the parser object's buffer_text attribute to True
will enable buffering.
The sample(population, k) function was
added to the random module. population is a sequence or
xrange object containing the elements of a population, and
sample() chooses k elements from the population without
replacing chosen elements. k can be any value up to
len(population). For example:
>>> days = ['Mo', 'Tu', 'We', 'Th', 'Fr', 'St', 'Sn']
>>> random.sample(days, 3) # Choose 3 elements
['St', 'Sn', 'Th']
>>> random.sample(days, 7) # Choose 7 elements
['Tu', 'Th', 'Mo', 'We', 'St', 'Fr', 'Sn']
>>> random.sample(days, 7) # Choose 7 again
['We', 'Mo', 'Sn', 'Fr', 'Tu', 'St', 'Th']
>>> random.sample(days, 8) # Can't choose eight
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "random.py", line 414, in sample
raise ValueError, "sample larger than population"
ValueError: sample larger than population
>>> random.sample(xrange(1,10000,2), 10) # Choose ten odd nos. under 10000
[3407, 3805, 1505, 7023, 2401, 2267, 9733, 3151, 8083, 9195]
The random module now uses a new algorithm, the Mersenne
Twister, implemented in C. It's faster and more extensively studied
than the previous algorithm.
(All changes contributed by Raymond Hettinger.)
The readline module also gained a number of new
functions: get_history_item(),
get_current_history_length(), and redisplay().
The rexec and Bastion modules have been
declared dead, and attempts to import them will fail with a
RuntimeError. New-style classes provide new ways to break
out of the restricted execution environment provided by
rexec, and no one has interest in fixing them or time to do
so. If you have applications using rexec, rewrite them to
use something else.
(Sticking with Python 2.2 or 2.1 will not make your applications any
safer because there are known bugs in the rexec module in
those versions. To repeat: if you're using rexec, stop using
it immediately.)
The rotor module has been deprecated because the
algorithm it uses for encryption is not believed to be secure. If
you need encryption, use one of the several AES Python modules
that are available separately.
The shutil module gained a move(src,
dest) function that recursively moves a file or directory to a new
location.
Support for more advanced POSIX signal handling was added
to the signal but then removed again as it proved impossible
to make it work reliably across platforms.
The socket module now supports timeouts. You
can call the settimeout(t) method on a socket object to
set a timeout of t seconds. Subsequent socket operations that
take longer than t seconds to complete will abort and raise a
socket.timeout exception.
The original timeout implementation was by Tim O'Malley. Michael
Gilfix integrated it into the Python socket module and
shepherded it through a lengthy review. After the code was checked
in, Guido van Rossum rewrote parts of it. (This is a good example of
a collaborative development process in action.)
On Windows, the socket module now ships with Secure
Sockets Layer (SSL) support.
The value of the C PYTHON_API_VERSION macro is now
exposed at the Python level as sys.api_version. The current
exception can be cleared by calling the new sys.exc_clear()
function.
The new tarfile module
allows reading from and writing to tar-format archive files.
(Contributed by Lars Gustäbel.)
The new textwrap module contains functions for wrapping
strings containing paragraphs of text. The wrap(text,
width) function takes a string and returns a list containing
the text split into lines of no more than the chosen width. The
fill(text, width) function returns a single
string, reformatted to fit into lines no longer than the chosen width.
(As you can guess, fill() is built on top of
wrap(). For example:
>>> import textwrap
>>> paragraph = "Not a whit, we defy augury: ... more text ..."
>>> textwrap.wrap(paragraph, 60)
["Not a whit, we defy augury: there's a special providence in",
"the fall of a sparrow. If it be now, 'tis not to come; if it",
...]
>>> print textwrap.fill(paragraph, 35)
Not a whit, we defy augury: there's
a special providence in the fall of
a sparrow. If it be now, 'tis not
to come; if it be not to come, it
will be now; if it be not now, yet
it will come: the readiness is all.
>>>
The module also contains a TextWrapper class that actually
implements the text wrapping strategy. Both the
TextWrapper class and the wrap() and
fill() functions support a number of additional keyword
arguments for fine-tuning the formatting; consult the module's
documentation for details.
(Contributed by Greg Ward.)
The thread and threading modules now have
companion modules, dummy_thread and dummy_threading,
that provide a do-nothing implementation of the thread
module's interface for platforms where threads are not supported. The
intention is to simplify thread-aware modules (ones that don't
rely on threads to run) by putting the following code at the top:
try:
import threading as _threading
except ImportError:
import dummy_threading as _threading
In this example, _threading is used as the module name to make
it clear that the module being used is not necessarily the actual
threading module. Code can call functions and use classes in
_threading whether or not threads are supported, avoiding an
if statement and making the code slightly clearer. This
module will not magically make multithreaded code run without threads;
code that waits for another thread to return or to do something will
simply hang forever.
The time module's strptime() function has
long been an annoyance because it uses the platform C library's
strptime() implementation, and different platforms
sometimes have odd bugs. Brett Cannon contributed a portable
implementation that's written in pure Python and should behave
identically on all platforms.
The new timeit module helps measure how long snippets
of Python code take to execute. The timeit.py file can be run
directly from the command line, or the module's Timer class
can be imported and used directly. Here's a short example that
figures out whether it's faster to convert an 8-bit string to Unicode
by appending an empty Unicode string to it or by using the
unicode() function:
import timeit
timer1 = timeit.Timer('unicode("abc")')
timer2 = timeit.Timer('"abc" + u""')
# Run three trials
print timer1.repeat(repeat=3, number=100000)
print timer2.repeat(repeat=3, number=100000)
# On my laptop this outputs:
# [0.36831796169281006, 0.37441694736480713, 0.35304892063140869]
# [0.17574405670166016, 0.18193507194519043, 0.17565798759460449]
The Tix module has received various bug fixes and
updates for the current version of the Tix package.
The Tkinter module now works with a thread-enabled
version of Tcl. Tcl's threading model requires that widgets only be
accessed from the thread in which they're created; accesses from
another thread can cause Tcl to panic. For certain Tcl interfaces,
Tkinter will now automatically avoid this
when a widget is accessed from a different thread by marshalling a
command, passing it to the correct thread, and waiting for the
results. Other interfaces can't be handled automatically but
Tkinter will now raise an exception on such an access so that
you can at least find out about the problem. See
http://mail.python.org/pipermail/python-dev/2002-December/031107.html for a more detailed explanation of this change. (Implemented by
Martin von Löwis.)
Calling Tcl methods through _tkinter no longer
returns only strings. Instead, if Tcl returns other objects those
objects are converted to their Python equivalent, if one exists, or
wrapped with a _tkinter.Tcl_Obj object if no Python equivalent
exists. This behavior can be controlled through the
wantobjects() method of tkapp objects.
When using _tkinter through the Tkinter module (as
most Tkinter applications will), this feature is always activated. It
should not cause compatibility problems, since Tkinter would always
convert string results to Python types where possible.
If any incompatibilities are found, the old behavior can be restored
by setting the wantobjects variable in the Tkinter
module to false before creating the first tkapp object.
import Tkinter
Tkinter.wantobjects = 0
Any breakage caused by this change should be reported as a bug.
The UserDict module has a new DictMixin class which
defines all dictionary methods for classes that already have a minimum
mapping interface. This greatly simplifies writing classes that need
to be substitutable for dictionaries, such as the classes in
the shelve module.
Adding the mix-in as a superclass provides the full dictionary
interface whenever the class defines __getitem__,
__setitem__, __delitem__, and keys.
For example:
The DOM implementation
in xml.dom.minidom can now generate XML output in a
particular encoding by providing an optional encoding argument to
the toxml() and toprettyxml() methods of DOM nodes.
The xmlrpclib module now supports an XML-RPC extension
for handling nil data values such as Python's None. Nil values
are always supported on unmarshalling an XML-RPC response. To
generate requests containing None, you must supply a true value
for the allow_none parameter when creating a Marshaller
instance.
The new DocXMLRPCServer module allows writing
self-documenting XML-RPC servers. Run it in demo mode (as a program)
to see it in action. Pointing the Web browser to the RPC server
produces pydoc-style documentation; pointing xmlrpclib to the
server allows invoking the actual methods.
(Contributed by Brian Quinlan.)
Support for internationalized domain names (RFCs 3454, 3490,
3491, and 3492) has been added. The ``idna'' encoding can be used
to convert between a Unicode domain name and the ASCII-compatible
encoding (ACE) of that name.
The socket module has also been extended to transparently
convert Unicode hostnames to the ACE version before passing them to
the C library. Modules that deal with hostnames such as
httplib and ftplib) also support Unicode host names;
httplib also sends HTTP "Host" headers using the ACE
version of the domain name. urllib supports Unicode URLs
with non-ASCII host names as long as the path part of the URL
is ASCII only.
To implement this change, the stringprep module, the
mkstringprep tool and the punycode encoding have been added.
Date and time types suitable for expressing timestamps were added as
the datetime module. The types don't support different
calendars or many fancy features, and just stick to the basics of
representing time.
The three primary types are: date, representing a day, month,
and year; time, consisting of hour, minute, and second; and
datetime, which contains all the attributes of both
date and time. There's also a
timedelta class representing differences between two points
in time, and time zone logic is implemented by classes inheriting from
the abstract tzinfo class.
You can create instances of date and time by either
supplying keyword arguments to the appropriate constructor,
e.g. datetime.date(year=1972, month=10, day=15), or by using
one of a number of class methods. For example, the date.today()
class method returns the current local date.
Once created, instances of the date/time classes are all immutable.
There are a number of methods for producing formatted strings from
objects:
>>> import datetime
>>> now = datetime.datetime.now()
>>> now.isoformat()
'2002-12-30T21:27:03.994956'
>>> now.ctime() # Only available on date, datetime
'Mon Dec 30 21:27:03 2002'
>>> now.strftime('%Y %d %b')
'2002 30 Dec'
The replace() method allows modifying one or more fields
of a date or datetime instance, returning a new instance:
Instances can be compared, hashed, and converted to strings (the
result is the same as that of isoformat()). date and
datetime instances can be subtracted from each other, and
added to timedelta instances. The largest missing feature is
that there's no standard library support for parsing strings and getting back a
date or datetime.
The getopt module provides simple parsing of command-line
arguments. The new optparse module (originally named Optik)
provides more elaborate command-line parsing that follows the Unix
conventions, automatically creates the output for --help,
and can perform different actions for different options.
You start by creating an instance of OptionParser and telling
it what your program's options are.
import sys
from optparse import OptionParser
op = OptionParser()
op.add_option('-i', '--input',
action='store', type='string', dest='input',
help='set input filename')
op.add_option('-l', '--length',
action='store', type='int', dest='length',
help='set maximum length of output')
Parsing a command line is then done by calling the parse_args()
method.
This returns an object containing all of the option values,
and a list of strings containing the remaining arguments.
Invoking the script with the various arguments now works as you'd
expect it to. Note that the length argument is automatically
converted to an integer.
$ ./python opt.py -i data arg1
<Values at 0x400cad4c: {'input': 'data', 'length': None}>
['arg1']
$ ./python opt.py --input=data --length=4
<Values at 0x400cad2c: {'input': 'data', 'length': 4}>
[]
$
The help message is automatically generated for you:
$ ./python opt.py --help
usage: opt.py [options]
options:
-h, --help show this help message and exit
-iINPUT, --input=INPUT
set input filename
-lLENGTH, --length=LENGTH
set maximum length of output
$
