Using DataStore in LabOne Q¶
LabOne Q comes with a storage interface to make it easier to store and retrieve any data or experiments. It currently supports the SQLite implementation native to Python.
In this notebook, you will learn how to:
- Create and connect to a database
- Save LabOne Q data to your database
- Access your stored data
- Delete data from a database
0. Python imports¶
import datetime
import io
import matplotlib.pyplot as plt
# pillow is currently required for displaying images after saving them to the database
from PIL import Image as PILImage
# LabOne Q:
from laboneq.simple import *
1. Creating and connecting to a database¶
You'll begin by creating a new database or connecting to an existing database at the default location. This location is ./laboneq_data/data.db
my_db = DataStore()
You can also create a new database or connect to an existing database at a customized location. You'll now do this in the path you define below.
custom_db_path = "laboneq_data/custom_database.db"
my_custom_db = DataStore(custom_db_path)
2. Saving a LabOne Q data object in the database¶
Before saving, you can check if the database already contains any data.
[my_db.get(k, with_metadata=True) for k in my_db.keys()]
You'll now create an Experiment and store it in the database, using the key "my_experiment"
. Not the the key can be different than the experiment uid
; you'll do this later on.
exp = Experiment("my_experiment", signals=[ExperimentSignal(uid="signal_1")])
my_db.store(exp, key="my_experiment")
You'll now store the same experiment as above using the same key. You'll now also include additional metadata for easier indexing and retrieval later. Note that this overwrites the previously stored dataset, since the same key is used.
my_db.store(
exp,
key="my_experiment",
metadata={
"author": "John Doe",
"creation_date": datetime.datetime.now(),
"setup": "CountZero",
},
)
Here, you'll store another version of the same experiment with a different key and different metadata.
my_db.store(
exp,
key="my_old_experiment",
metadata={
"author": "John Doe",
"creation_date": datetime.datetime(year=2021, month=4, day=20),
"setup": "CountZero",
},
)
Images from your data analysis can also be saved as part of the metadata for quick access. You'll now generate a plot using matplotlib.pyplot
below, and then store it as part of the database metadata.
plt.plot([1, 2, 3, 4])
plt.ylabel("It's linear!")
# save as png
img_buf = io.BytesIO()
plt.savefig(img_buf, format="png")
image_bytes = img_buf.getvalue()
# include the figure as part of the metadata
my_db.store(
exp,
key="experiment_with_image",
metadata={
"result_plot_png_bytes": image_bytes,
"creation_date": datetime.datetime.now(),
},
)
3. Accessing the stored data by key and through metadata¶
You'll start by checking if the database contains any data.
[my_db.get(k, with_metadata=True) for k in my_db.keys()]
You'll then access the list of available keys.
list(my_db.keys())
You'll then access the metadata for each available key.
for k in my_db.keys():
print(k, my_db.get_metadata(k))
If you want to find and access only specific datasets, selected by metadata fields, you can do that as well. Let's say you have three experimental set-ups, named for the Sprawl Trilogy. You'd only like to access one of them, the data associated with Count Zero:
count_zero_keys = my_db.find(metadata={"setup_name": "CountZero"})
for k in count_zero_keys:
print(k, my_db.get(k, with_metadata=True))
You may, instead, want to perform a more general query of the metadata based on the creation date:
new_data = my_db.find(
condition=lambda metadata: (
"creation_date" in metadata
and metadata["creation_date"] > datetime.datetime(year=2023, month=4, day=20)
)
)
for k in new_data:
print(k, my_db.get(k, with_metadata=True))
You can also display an image saved in the metadata.
im = PILImage.open(
io.BytesIO(my_db.get_metadata("experiment_with_image")["result_plot_png_bytes"])
)
# Note: The im.convert is a temporary workaround for a bug in Pillow 10.0.0
# that is fixed in https://github.com/python-pillow/Pillow/pull/7266 but
# not released as of 5 September 2023.
im = im.convert("RGB")
im
4. Deleting data from the database¶
Finally, you may want to delete entries from your database. You can do this in the following way:
my_db.delete("experiment_with_image")
my_db.delete("my_experiment")
my_db.delete("my_old_experiment")
You've now created, accessed, and deleted data in a database with LabOne Q.
Have more questions about incorporating LabOne Q in your experiments? Contact us at info@zhinst.com.