Descriptor Remote¶
Module for remotely retrieving descriptors from directory authorities and mirrors. This is the simplest method for getting current tor descriptor information…
import stem.descriptor.remote
for desc in stem.descriptor.remote.get_server_descriptors():
if desc.exit_policy.is_exiting_allowed():
print(' %s (%s)' % (desc.nickname, desc.fingerprint))
More custom downloading behavior can be done through the
DescriptorDownloader
class, which issues
Query
instances to get you descriptor
content. For example…
from stem.descriptor.remote import DescriptorDownloader
downloader = DescriptorDownloader(
use_mirrors = True,
timeout = 10,
)
query = downloader.get_server_descriptors()
print('Exit Relays:')
try:
for desc in query.run():
if desc.exit_policy.is_exiting_allowed():
print(' %s (%s)' % (desc.nickname, desc.fingerprint))
print
print('Query took %0.2f seconds' % query.runtime)
except Exception as exc:
print('Unable to retrieve the server descriptors: %s' % exc)
get_instance - Provides a singleton DescriptorDownloader used for...
|- their_server_descriptor - provides the server descriptor of the relay we download from
|- get_server_descriptors - provides present server descriptors
|- get_extrainfo_descriptors - provides present extrainfo descriptors
|- get_microdescriptors - provides present microdescriptors with the given digests
|- get_consensus - provides the present consensus or router status entries
|- get_bandwidth_file - provides bandwidth heuristics used to make the next consensus
+- get_detached_signatures - authority signatures used to make the next consensus
Query - Asynchronous request to download tor descriptors
|- start - issues the query if it isn't already running
+- run - blocks until the request is finished and provides the results
DescriptorDownloader - Configurable class for issuing queries
|- use_directory_mirrors - use directory mirrors to download future descriptors
|- their_server_descriptor - provides the server descriptor of the relay we download from
|- get_server_descriptors - provides present server descriptors
|- get_extrainfo_descriptors - provides present extrainfo descriptors
|- get_microdescriptors - provides present microdescriptors with the given digests
|- get_consensus - provides the present consensus or router status entries
|- get_vote - provides an authority's vote for the next consensus
|- get_key_certificates - provides present authority key certificates
|- get_bandwidth_file - provides bandwidth heuristics used to make the next consensus
|- get_detached_signatures - authority signatures used to make the next consensus
+- query - request an arbitrary descriptor resource
New in version 1.1.0.
-
stem.descriptor.remote.
MAX_FINGERPRINTS
¶ Maximum number of descriptors that can requested at a time by their fingerprints.
-
stem.descriptor.remote.
MAX_MICRODESCRIPTOR_HASHES
¶ Maximum number of microdescriptors that can requested at a time by their hashes.
-
stem.descriptor.remote.
Compression
(enum)¶ Compression when downloading descriptors.
New in version 1.7.0.
Compression
Description
PLAINTEXT
Uncompressed data.
GZIP
ZSTD
Zstandard compression, this requires the zstandard module.
LZMA
LZMA compression, this requires the ‘lzma module <https://docs.python.org/3/library/lzma.html>`_.
-
stem.descriptor.remote.
get_instance
()[source]¶ Provides the singleton
DescriptorDownloader
used for this module’s shorthand functions.New in version 1.5.0.
- Returns
singleton
DescriptorDownloader
instance
-
stem.descriptor.remote.
their_server_descriptor
(**query_args)[source]¶ Provides the server descriptor of the relay we’re downloading from.
New in version 1.7.0.
-
stem.descriptor.remote.
get_server_descriptors
(fingerprints=None, **query_args)[source]¶ Shorthand for
get_server_descriptors()
on our singleton instance.New in version 1.5.0.
-
stem.descriptor.remote.
get_extrainfo_descriptors
(fingerprints=None, **query_args)[source]¶ Shorthand for
get_extrainfo_descriptors()
on our singleton instance.New in version 1.5.0.
-
stem.descriptor.remote.
get_microdescriptors
(hashes, **query_args)[source]¶ Shorthand for
get_microdescriptors()
on our singleton instance.New in version 1.8.0.
-
stem.descriptor.remote.
get_consensus
(authority_v3ident=None, microdescriptor=False, **query_args)[source]¶ Shorthand for
get_consensus()
on our singleton instance.New in version 1.5.0.
-
stem.descriptor.remote.
get_bandwidth_file
(**query_args)[source]¶ Shorthand for
get_bandwidth_file()
on our singleton instance.New in version 1.8.0.
-
stem.descriptor.remote.
get_detached_signatures
(**query_args)[source]¶ Shorthand for
get_detached_signatures()
on our singleton instance.New in version 1.8.0.
-
class
stem.descriptor.remote.
Query
(resource, descriptor_type=None, endpoints=None, compression=('gzip', ), retries=2, fall_back_to_authority=False, timeout=None, start=True, block=False, validate=False, document_handler='ENTRIES', **kwargs)[source]¶ Bases:
object
Asynchronous request for descriptor content from a directory authority or mirror. These can either be made through the
DescriptorDownloader
or directly for more advanced usage.To block on the response and get results either call
run()
or iterate over the Query. Therun()
method pass along any errors that arise…from stem.descriptor.remote import Query query = Query( '/tor/server/all', timeout = 30, ) print('Current relays:') try: for desc in Query('/tor/server/all', 'server-descriptor 1.0').run(): print(desc.fingerprint) except Exception as exc: print('Unable to retrieve the server descriptors: %s' % exc)
… while iterating fails silently…
print('Current relays:') for desc in Query('/tor/server/all', 'server-descriptor 1.0'): print(desc.fingerprint)
In either case exceptions are available via our ‘error’ attribute.
Tor provides quite a few different descriptor resources via its directory protocol (see section 4.2 and later of the dir-spec). Commonly useful ones include…
Resource
Description
/tor/server/all
all present server descriptors
/tor/server/fp/<fp1>+<fp2>+<fp3>
server descriptors with the given fingerprints
/tor/extra/all
all present extrainfo descriptors
/tor/extra/fp/<fp1>+<fp2>+<fp3>
extrainfo descriptors with the given fingerprints
/tor/micro/d/<hash1>-<hash2>
microdescriptors with the given hashes
/tor/status-vote/current/consensus
present consensus
/tor/status-vote/current/consensus-microdesc
present microdescriptor consensus
/tor/status-vote/next/bandwidth
bandwidth authority heuristics for the next consenus
/tor/status-vote/next/consensus-signatures
detached signature, used for making the next consenus
/tor/keys/all
key certificates for the authorities
/tor/keys/fp/<v3ident1>+<v3ident2>
key certificates for specific authorities
ZSTD compression requires zstandard, and LZMA requires the lzma module.
For legacy reasons if our resource has a ‘.z’ suffix then our compression argument is overwritten with Compression.GZIP.
Changed in version 1.7.0: Added support for downloading from ORPorts.
Changed in version 1.7.0: Added the compression argument.
Changed in version 1.7.0: Added the reply_headers attribute.
The class this provides changed between Python versions. In python2 this was called httplib.HTTPMessage, whereas in python3 the class was renamed to http.client.HTTPMessage.
Changed in version 1.7.0: Endpoints are now expected to be
DirPort
orORPort
instances. Usage of tuples for this argument is deprecated and will be removed in the future.Changed in version 1.7.0: Avoid downloading from tor26. This directory authority throttles its DirPort to such an extent that requests either time out or take on the order of minutes.
Changed in version 1.7.0: Avoid downloading from Bifroest. This is the bridge authority so it doesn’t vote in the consensus, and apparently times out frequently.
Changed in version 1.8.0: Serge has replaced Bifroest as our bridge authority. Avoiding descriptor downloads from it instead.
Changed in version 1.8.0: Defaulting to gzip compression rather than plaintext downloads.
Changed in version 1.8.0: Using
Compression
for our compression argument, usage of strings or this module’s Compression enum is deprecated and will be removed in stem 2.x.- Variables
resource (str) – resource being fetched, such as ‘/tor/server/all’
descriptor_type (str) – type of descriptors being fetched (for options see
parse_file()
), this is guessed from the resource if Noneendpoints (list) –
DirPort
orORPort
of the authority or mirror we’re querying, this uses authorities if undefinedcompression (list) – list of
stem.descriptor.Compression
we’re willing to accept, when none are mutually supported downloads fall back to Compression.PLAINTEXTretries (int) – number of times to attempt the request if downloading it fails
fall_back_to_authority (bool) – when retrying request issues the last request to a directory authority if True
content (str) – downloaded descriptor content
error (Exception) – exception if a problem occured
is_done (bool) – flag that indicates if our request has finished
start_time (float) – unix timestamp when we first started running
reply_headers (http.client.HTTPMessage) – headers provided in the response, None if we haven’t yet made our request
runtime (float) – time our query took, this is None if it’s not yet finished
validate (bool) – checks the validity of the descriptor’s content if True, skips these checks otherwise
document_handler (stem.descriptor.__init__.DocumentHandler) – method in which to parse a
NetworkStatusDocument
kwargs (dict) – additional arguments for the descriptor constructor
Following are only applicable when downloading from a
DirPort
…- Variables
timeout (float) – duration before we’ll time out our request
download_url (str) – last url used to download the descriptor, this is unset until we’ve actually made a download attempt
- Parameters
start (bool) – start making the request when constructed (default is True)
block (bool) – only return after the request has been completed, this is the same as running query.run(True) (default is False)
-
run
(suppress=False)[source]¶ Blocks until our request is complete then provides the descriptors. If we haven’t yet started our request then this does so.
- Parameters
suppress (bool) – avoids raising exceptions if True
- Returns
list for the requested
Descriptor
instances- Raises
Using the iterator can fail with the following if suppress is False…
ValueError if the descriptor contents is malformed
DownloadTimeout
if our request timed outDownloadFailed
if our request fails
-
class
stem.descriptor.remote.
DescriptorDownloader
(use_mirrors=False, **default_args)[source]¶ Bases:
object
Configurable class that issues
Query
instances on your behalf.- Parameters
use_mirrors (bool) – downloads the present consensus and uses the directory mirrors to fetch future requests, this fails silently if the consensus cannot be downloaded
default_args – default arguments for the
Query
constructor
-
use_directory_mirrors
()[source]¶ Downloads the present consensus and configures ourselves to use directory mirrors, in addition to authorities.
- Returns
NetworkStatusDocumentV3
from which we got the directory mirrors- Raises
Exception if unable to determine the directory mirrors
-
their_server_descriptor
(**query_args)[source]¶ Provides the server descriptor of the relay we’re downloading from.
New in version 1.7.0.
-
get_server_descriptors
(fingerprints=None, **query_args)[source]¶ Provides the server descriptors with the given fingerprints. If no fingerprints are provided then this returns all descriptors known by the relay.
- Parameters
fingerprints (str,list) – fingerprint or list of fingerprints to be retrieved, gets all descriptors if None
query_args – additional arguments for the
Query
constructor
- Returns
Query
for the server descriptors- Raises
ValueError if we request more than 96 descriptors by their fingerprints (this is due to a limit on the url length by squid proxies).
-
get_extrainfo_descriptors
(fingerprints=None, **query_args)[source]¶ Provides the extrainfo descriptors with the given fingerprints. If no fingerprints are provided then this returns all descriptors in the present consensus.
- Parameters
fingerprints (str,list) – fingerprint or list of fingerprints to be retrieved, gets all descriptors if None
query_args – additional arguments for the
Query
constructor
- Returns
Query
for the extrainfo descriptors- Raises
ValueError if we request more than 96 descriptors by their fingerprints (this is due to a limit on the url length by squid proxies).
-
get_microdescriptors
(hashes, **query_args)[source]¶ Provides the microdescriptors with the given hashes. To get these see the microdescriptor_digest attribute of
RouterStatusEntryMicroV3
. Note that these are only provided via the microdescriptor consensus. For exampe…>>> import stem.descriptor.remote >>> consensus = stem.descriptor.remote.get_consensus(microdescriptor = True).run() >>> my_router_status_entry = list(filter(lambda desc: desc.nickname == 'caersidi', consensus))[0] >>> print(my_router_status_entry.microdescriptor_digest) IQI5X2A5p0WVN/MgwncqOaHF2f0HEGFEaxSON+uKRhU >>> my_microdescriptor = stem.descriptor.remote.get_microdescriptors([my_router_status_entry.microdescriptor_digest]).run()[0] >>> print(my_microdescriptor) onion-key -----BEGIN RSA PUBLIC KEY----- MIGJAoGBAOJo9yyVgG8ksEHQibqPIEbLieI6rh1EACRPiDiV21YObb+9QEHaR3Cf FNAzDbGhbvADLBB7EzuViL8w+eXQUOaIsJRdymh/wuUJ78bv5oEIJhthKq/Uqa4P wKHXSZixwAHfy8NASTX3kxu9dAHWU3Owb+4W4lR2hYM0ZpoYYkThAgMBAAE= -----END RSA PUBLIC KEY----- ntor-onion-key kWOHNd+2uBlMpcIUbbpFLiq/rry66Ep6MlwmNpwzcBg= id ed25519 xE/GeYImYAIB0RbzJXFL8kDLpDrj/ydCuCdvOgC4F/4
- Parameters
hashes (str,list) – microdescriptor hash or list of hashes to be retrieved
query_args – additional arguments for the
Query
constructor
- Returns
Query
for the microdescriptors- Raises
ValueError if we request more than 92 microdescriptors by their hashes (this is due to a limit on the url length by squid proxies).
-
get_consensus
(authority_v3ident=None, microdescriptor=False, **query_args)[source]¶ Provides the present router status entries.
Changed in version 1.5.0: Added the microdescriptor argument.
- Parameters
authority_v3ident (str) – fingerprint of the authority key for which to get the consensus, see ‘v3ident’ in tor’s config.c for the values.
microdescriptor (bool) – provides the microdescriptor consensus if True, standard consensus otherwise
query_args – additional arguments for the
Query
constructor
- Returns
Query
for the router status entries
-
get_vote
(authority, **query_args)[source]¶ Provides the present vote for a given directory authority.
- Parameters
authority (stem.directory.Authority) – authority for which to retrieve a vote for
query_args – additional arguments for the
Query
constructor
- Returns
Query
for the router status entries
-
get_key_certificates
(authority_v3idents=None, **query_args)[source]¶ Provides the key certificates for authorities with the given fingerprints. If no fingerprints are provided then this returns all present key certificates.
- Parameters
authority_v3idents (str) –
fingerprint or list of fingerprints of the authority keys, see ‘v3ident’ in tor’s config.c for the values.
query_args – additional arguments for the
Query
constructor
- Returns
Query
for the key certificates- Raises
ValueError if we request more than 96 key certificates by their identity fingerprints (this is due to a limit on the url length by squid proxies).
-
get_bandwidth_file
(**query_args)[source]¶ Provides the bandwidth authority heuristics used to make the next consensus.
New in version 1.8.0.
-
get_detached_signatures
(**query_args)[source]¶ Provides the detached signatures that will be used to make the next consensus. Please note that these are only available during minutes 55-60 each hour. If requested during minutes 0-55 tor will not service these requests, and this will fail with a 404.
For example…
import stem.descriptor.remote detached_sigs = stem.descriptor.remote.get_detached_signatures().run()[0] for i, sig in enumerate(detached_sigs.signatures): print('Signature %i is from %s' % (i + 1, sig.identity))
When available (minutes 55-60 of the hour)
% python demo.py Signature 1 is from 0232AF901C31A04EE9848595AF9BB7620D4C5B2E Signature 2 is from 14C131DFC5C6F93646BE72FA1401C02A8DF2E8B4 Signature 3 is from 23D15D965BC35114467363C165C4F724B64B4F66 ...
When unavailable (minutes 0-55 of the hour)
% python demo.py Traceback (most recent call last): File "demo.py", line 3, in detached_sigs = stem.descriptor.remote.get_detached_signatures().run()[0] File "/home/atagar/Desktop/stem/stem/descriptor/remote.py", line 533, in run return list(self._run(suppress)) File "/home/atagar/Desktop/stem/stem/descriptor/remote.py", line 544, in _run raise self.error stem.DownloadFailed: Failed to download from http://154.35.175.225:80/tor/status-vote/next/consensus-signatures (HTTPError): Not found
New in version 1.8.0.
Provides cached Tor directory authority information. The directory information hardcoded into Tor and occasionally changes, so the information this provides might not necessarily match your version of tor.
Deprecated since version 1.7.0: Use stem.directory.Authority.from_cache() instead.
- Returns
dict of str nicknames to
Authority
instances