Configuration

LUME-services uses injection for runtime configuration of services. This means that programs can use the same code in a number of different environments simply by setting environment variables.

The LUMEServicesSettings object

Applications that LUME-services tools (the MongoDB implementation of Results Database and MySQL model database), can use the configuration tools packaged with LUME-services directly by calling the configure script during code execution:

from lume_services.config import configure, LUMEServicesSettings
from lume_services.services.models.db import ModelDBConfig
from lume_services.services.results.mongodb import MongodbResultsDBConfig
from lume_services.services.scheduling.backends.server import (
    PrefectAgentConfig,
    PrefectConfig,
    PrefectServerConfig,
)
from lume_services import config

from lume_services.services.files.filesystems import (
    MountedFilesystem,
)

model_db_config = ModelDBConfig(
    host="127.0.0.1",
    port=3306,
    user="root",
    password="test",
    database="model_db",
)

results_db_config = MongodbResultsDBConfig(
    port=27017,
    host="127.0.0.1",
    username="root",
    database="model_db",
    password="test",
)

prefect_config = PrefectConfig(
    server=PrefectServerConfig(
        host="http://localhost", 
        host_port=4200, 
    ),
)

agent=PrefectAgentConfig(
  host="http://localhost", 
  host_port=5000,
  backend="server",
  debug=True,
)

mounted_filesystem = MountedFilesystem(
        mount_path="~/sandbox/lume-services",
        mount_alias="/User/my_user/data",
        identifier="mounted",
        mount_type="DirectoryOrCreate",
    )

settings = config.LUMEServicesSettings(
    model_db=model_db_config,
    results_db=results_db_config,
    prefect=prefect_config,
    backend="docker",
    mounted_filesystem=mounted_filesystem,
)

config.configure(settings)

Configurations are held on LUMEServicesSettings objects. For more information on how configurations are handled with injection, see developer/configuration.

Configure from environment

The LUME-services environment may alternatively be configured using environment variables by calling the configure method without an argument.

from lume_services.config import configure

configure()

Relevant environment variables (these tables are built using script in scripts/build_configuration_table.py automatically generated w/ the GitHub build action defined in .github/workflows/build_docs.yml, see **note beneath):

Base Configuration

Name	Type	Default
LUME_BACKEND	string	local

Filesystem Configuration

Name	Type	Default
LUME_MOUNTED_FILESYSTEM__IDENTIFIER	string	mounted
LUME_MOUNTED_FILESYSTEM__MOUNT_PATH	string
LUME_MOUNTED_FILESYSTEM__MOUNT_ALIAS	string
LUME_MOUNTED_FILESYSTEM__MOUNT_TYPE	string	DirectoryOrCreate

Model Database

Name	Type	Default
LUME_MODEL_DB__HOST	string
LUME_MODEL_DB__PORT	integer
LUME_MODEL_DB__USER	string
LUME_MODEL_DB__PASSWORD	string
LUME_MODEL_DB__DATABASE	string
LUME_MODEL_DB__CONNECTION__POOL_SIZE	integer
LUME_MODEL_DB__CONNECTION__POOL_PRE_PING	boolean	True
LUME_MODEL_DB__DIALECT_STR	string	mysql+pymysql

Results Database

Name	Type	Default
LUME_RESULTS_DB__DATABASE	string
LUME_RESULTS_DB__USERNAME	string
LUME_RESULTS_DB__HOST	string
LUME_RESULTS_DB__PASSWORD	string
LUME_RESULTS_DB__PORT	integer
LUME_RESULTS_DB__AUTHMECHANISM	string	DEFAULT
LUME_RESULTS_DB__OPTIONS	object	{}

Scheduling Service

Name	Type	Default
LUME_PREFECT__SERVER__TAG	string	core-1.4.0
LUME_PREFECT__SERVER__HOST	string	http://localhost
LUME_PREFECT__SERVER__HOST_PORT	string	4200
LUME_PREFECT__SERVER__HOST_IP	string	127.0.0.1
LUME_PREFECT__UI__HOST	string	http://localhost
LUME_PREFECT__UI__HOST_PORT	string	8080
LUME_PREFECT__UI__HOST_IP	string	127.0.0.1
LUME_PREFECT__UI__APOLLO_URL	string	http://localhost:4200/graphql
LUME_PREFECT__TELEMETRY__ENABLED	boolean	True
LUME_PREFECT__AGENT__HOST	string	http://localhost
LUME_PREFECT__AGENT__HOST_PORT	string	5000
LUME_PREFECT__HOME_DIR	string	~/.prefect
LUME_PREFECT__DEBUG	boolean	False
LUME_PREFECT__BACKEND	string	server
LUME_PREFECT__IMAGE	string	jgarrahan/lume-services-prefect:latest
LUME_PREFECT__ISOLATED	boolean	False

**note: I haven't validated these table. Sorry! You can validate the fields here, which are constructed using Pydantic's _env_nested_delimiter configuration setting.

Custom configuration

The lume-services.config.configure method provides tooling for one specific architecture configuration, but other users may want to use different database implementations (see services). These applications can define their own methods using the same interface by subclassing the base class of artifacts that is injected into each service:

base class -> service injected into:

• lume_services.services.files.filesystems.filesystem -> lume_services.services.files.service
• lume_services.services.results.db -> lume_services.services.results.service
• lume_services.services.models.db.db -> lume_services.services.models.services
• lume_services.services.scheduling.backends.backend -> lume_services.services.scheduling.service
• (Downstream, for things like slurm scheduling) lume_services.hpc.provider -> lume_services.services.hpc.service