saver package

saver.saver_class module

class saver.saver_class.Saver(database_url)

Bases: object

A wrapper class for the saver_agents package. Recieves a database_url parameter, determines which saver to use based on the scheme

save(data, field)

Saves the data of the given snapshot and the respective field.

Parameters

  • data – Parsed data dict as received from the parser of the respective field

  • field – The field of the parser from which the data was received

Module contents

cortex.saver is responsible for saving parsed results to the database. Saver can be run as a service or a CLI command

CLI

python -m cortex.saver

python -m cortex.saver [OPTIONS] COMMAND [ARGS]...

run-saver

Receives urls (+scheme) to a database and a message queue and runs the saver service, which listens to the queue and saves message to the database

python -m cortex.saver run-saver [OPTIONS] DATABASE MQ

Arguments

DATABASE

Required argument

MQ

Required argument

save

Receives a path to a result message (as received from a parser) file and a field name, saves the result to the specified database

python -m cortex.saver save [OPTIONS] FIELD RESULT_PATH

Options

-d, --database <database>

The url (+scheme) of the database to save the result in

Arguments

FIELD

Required argument

RESULT_PATH

Required argument