Metadata#
Metadata can be added to TsGroup, IntervalSet, and TsdFrame objects at initialization or after an object has been created.
TsGroupmetadata is information associated with each Ts/Tsd object, such as brain region or unit type.IntervalSetmetadata is information associated with each interval, such as a trial label or stimulus condition.TsdFramemetadata is information associated with each column, such as a channel or position.
Adding metadata at initialization#
At initialization, metadata can be passed via a dictionary or pandas DataFrame using the keyword argument metadata. The metadata name is taken from the dictionary key or DataFrame column, and it can be set to any string name with a couple class-specific exceptions.
Class-specific exceptions
If column names are supplied to
TsdFrame, metadata cannot overlap with those names.The
rateattribute forTsGroupis stored with the metadata and cannot be overwritten.
The length of the metadata must match the length of the object it describes (see class examples below for more detail).
Show code cell content
import numpy as np
import pandas as pd
import pynapple as nap
# input parameters for TsGroup
group = {
1: nap.Ts(t=np.sort(np.random.uniform(0, 100, 100))),
2: nap.Ts(t=np.sort(np.random.uniform(0, 100, 200))),
3: nap.Ts(t=np.sort(np.random.uniform(0, 100, 300))),
4: nap.Ts(t=np.sort(np.random.uniform(0, 100, 400))),
}
# input parameters for IntervalSet
starts = [0,35,70]
ends = [30,65,100]
# input parameters for TsdFrame
t = np.arange(5)
d = np.tile([1,2,3], (5, 1))
columns = ["a", "b", "c"]
TsGroup#
Metadata added to TsGroup must match the number of Ts/Tsd objects, or the length of its index property.
metadata = {"region": ["pfc", "pfc", "hpc", "hpc"]}
tsgroup = nap.TsGroup(group, metadata=metadata)
print(tsgroup)
Index rate region
------- ------- --------
1 1.0011 pfc
2 2.00219 pfc
3 3.00329 hpc
4 4.00439 hpc
When initializing with a DataFrame, the index must align with the input dictionary keys (only when a dictionary is used to create the TsGroup).
metadata = pd.DataFrame(
index=group.keys(),
data=["pfc", "pfc", "hpc", "hpc"],
columns=["region"]
)
tsgroup = nap.TsGroup(group, metadata=metadata)
print(tsgroup)
Index rate region
------- ------- --------
1 1.0011 pfc
2 2.00219 pfc
3 3.00329 hpc
4 4.00439 hpc
IntervalSet#
Metadata added to IntervalSet must match the number of intervals, or the length of its index property.
metadata = {
"reward": [1, 0, 1],
"choice": ["left", "right", "left"],
}
intervalset = nap.IntervalSet(starts, ends, metadata=metadata)
print(intervalset)
index start end reward choice
0 0 30 1 left
1 35 65 0 right
2 70 100 1 left
shape: (3, 2), time unit: sec.
Metadata can be initialized as a DataFrame using the metadata argument, or it can be inferred when initializing an IntervalSet with a DataFrame.
df = pd.DataFrame(
data=[[0, 30, 1, "left"], [35, 65, 0, "right"], [70, 100, 1, "left"]],
columns=["start", "end", "reward", "choice"]
)
intervalset = nap.IntervalSet(df)
print(intervalset)
index start end reward choice
0 0 30 1 left
1 35 65 0 right
2 70 100 1 left
shape: (3, 2), time unit: sec.
TsdFrame#
Metadata added to TsdFrame must match the number of data columns, or the length of its columns property.
metadata = {
"color": ["red", "blue", "green"],
"position": [10,20,30],
"label": ["x", "x", "y"]
}
tsdframe = nap.TsdFrame(d=d, t=t, columns=["a", "b", "c"], metadata=metadata)
print(tsdframe)
Time (s) a b c
---------- --- ---- -----
0.0 1 2 3
1.0 1 2 3
2.0 1 2 3
3.0 1 2 3
4.0 1 2 3
Metadata
color red blue green
position 10 20 30
... ... ... ...
dtype: int64, shape: (5, 3)
When initializing with a DataFrame, the DataFrame index must match the TsdFrame columns.
metadata = pd.DataFrame(
index=["a", "b", "c"],
data=[["red", 10, "x"], ["blue", 20, "x"], ["green", 30, "y"]],
columns=["color", "position", "label"],
)
tsdframe = nap.TsdFrame(d=d, t=t, columns=["a", "b", "c"], metadata=metadata)
print(tsdframe)
Time (s) a b c
---------- --- ---- -----
0.0 1 2 3
1.0 1 2 3
2.0 1 2 3
3.0 1 2 3
4.0 1 2 3
Metadata
color red blue green
position 10 20 30
... ... ... ...
dtype: int64, shape: (5, 3)
Adding metadata after initialization#
After creation, metadata can be added using the class method set_info(). Additionally, single metadata fields can be added as a dictionary-like key or as an attribute, with a few noted exceptions outlined below.
Note
The remaining metadata examples will be shown on a TsGroup object; however, all examples can be directly applied to IntervalSet and TsdFrame objects.
set_info#
Metadata can be passed as a dictionary or pandas DataFrame as the first positional argument, or metadata can be passed as name-value keyword arguments.
tsgroup.set_info(unit_type=["multi", "single", "single", "single"])
print(tsgroup)
Index rate region unit_type
------- ------- -------- -----------
1 1.0011 pfc multi
2 2.00219 pfc single
3 3.00329 hpc single
4 4.00439 hpc single
Using dictionary-like keys (square brackets)#
Most metadata names can set as a dictionary-like key (i.e. using square brackets). The only exceptions are for IntervalSet, where the names “start” and “end” are reserved for class properties.
tsgroup["depth"] = [0, 1, 2, 3]
print(tsgroup)
Index rate region unit_type depth
------- ------- -------- ----------- -------
1 1.0011 pfc multi 0
2 2.00219 pfc single 1
3 3.00329 hpc single 2
4 4.00439 hpc single 3
Using attribute assignment#
If the metadata name is unique from other class attributes and methods, and it is formatted properly (i.e. only alpha-numeric characters and underscores), it can be set as an attribute (i.e. using a . followed by the metadata name).
tsgroup.label=["MUA", "good", "good", "good"]
print(tsgroup)
Index rate region unit_type depth label
------- ------- -------- ----------- ------- -------
1 1.0011 pfc multi 0 MUA
2 2.00219 pfc single 1 good
3 3.00329 hpc single 2 good
4 4.00439 hpc single 3 good
Allowed data types#
As long as the length of the metadata container matches the length of the object (number of columns for TsdFrame and number of indices for IntervalSet and TsGroup), elements of the metadata can be any data type.
tsgroup.coords = [[1,0],[0,1],[1,1],[2,1]]
print(tsgroup)
Index rate region unit_type depth label coords ...
------- ------- -------- ----------- ------- ------- -------- -----
1 1.0011 pfc multi 0 MUA [1 0] ...
2 2.00219 pfc single 1 good [0 1] ...
3 3.00329 hpc single 2 good [1 1] ...
4 4.00439 hpc single 3 good [2 1] ...
Accessing metadata#
Metadata is stored as a pandas DataFrame, which can be previewed using the metadata attribute.
print(tsgroup.metadata)
rate region unit_type depth label coords
1 1.001096 pfc multi 0 MUA [1, 0]
2 2.002193 pfc single 1 good [0, 1]
3 3.003289 hpc single 2 good [1, 1]
4 4.004385 hpc single 3 good [2, 1]
Single metadata columns (or lists of columns) can be retrieved using the get_info() class method:
print(tsgroup.get_info("region"))
1 pfc
2 pfc
3 hpc
4 hpc
Name: region, dtype: object
Similarly, metadata can be accessed using key indexing (i.e. square brakets)
print(tsgroup["region"])
1 pfc
2 pfc
3 hpc
4 hpc
Name: region, dtype: object
Note
Metadata names must be strings. Key indexing with an integer will produce different behavior based on object type.
Finally, metadata that can be set as an attribute can also be accessed as an attribute.
print(tsgroup.region)
1 pfc
2 pfc
3 hpc
4 hpc
Name: region, dtype: object
Overwriting metadata#
User-set metadata is mutable and can be overwritten.
print(tsgroup, "\n")
tsgroup.set_info(label=["A", "B", "C", "D"])
print(tsgroup)
Index rate region unit_type depth label coords ...
------- ------- -------- ----------- ------- ------- -------- -----
1 1.0011 pfc multi 0 MUA [1 0] ...
2 2.00219 pfc single 1 good [0 1] ...
3 3.00329 hpc single 2 good [1 1] ...
4 4.00439 hpc single 3 good [2 1] ...
Index rate region unit_type depth label coords ...
------- ------- -------- ----------- ------- ------- -------- -----
1 1.0011 pfc multi 0 A [1 0] ...
2 2.00219 pfc single 1 B [0 1] ...
3 3.00329 hpc single 2 C [1 1] ...
4 4.00439 hpc single 3 D [2 1] ...
Dropping metadata#
To drop metadata, use the drop_info() method. Multiple metadata columns can be dropped by passing a list of metadata names.
print(tsgroup, "\n")
tsgroup.drop_info("coords")
print(tsgroup)
Index rate region unit_type depth label coords ...
------- ------- -------- ----------- ------- ------- -------- -----
1 1.0011 pfc multi 0 A [1 0] ...
2 2.00219 pfc single 1 B [0 1] ...
3 3.00329 hpc single 2 C [1 1] ...
4 4.00439 hpc single 3 D [2 1] ...
Index rate region unit_type depth label
------- ------- -------- ----------- ------- -------
1 1.0011 pfc multi 0 A
2 2.00219 pfc single 1 B
3 3.00329 hpc single 2 C
4 4.00439 hpc single 3 D
Using metadata to slice objects#
Metadata can be used to slice or filter objects based on metadata values.
print(tsgroup[tsgroup.label == "A"])
Index rate region unit_type depth label
------- ------ -------- ----------- ------- -------
1 1.0011 pfc multi 0 A
groupby: Using metadata to group objects#
Similar to pandas, metadata can be used to group objects based on one or more metadata columns using the object method groupby, where the first argument is the metadata columns name(s) to group by. This function returns a dictionary with keys corresponding to unique groups and values corresponding to object indices belonging to each group.
print(tsgroup,"\n")
print(tsgroup.groupby("region"))
Index rate region unit_type depth label
------- ------- -------- ----------- ------- -------
1 1.0011 pfc multi 0 A
2 2.00219 pfc single 1 B
3 3.00329 hpc single 2 C
4 4.00439 hpc single 3 D
{'hpc': array([3, 4]), 'pfc': array([1, 2])}
Grouping by multiple metadata columns should be passed as a list.
tsgroup.groupby(["region","unit_type"])
{('hpc', 'single'): array([3, 4]),
('pfc', 'multi'): array([1]),
('pfc', 'single'): array([2])}
The optional argument get_group can be provided to return a new object corresponding to a specific group.
tsgroup.groupby("region", get_group="hpc")
Index rate region unit_type depth label
------- ------- -------- ----------- ------- -------
3 3.00329 hpc single 2 C
4 4.00439 hpc single 3 D
groupby_apply: Applying functions to object groups#
The groupby_apply object method allows a specific function to be applied to object groups. The first argument, same as groupby, is the metadata column(s) used to group the object. The second argument is the function to apply to each group. If only these two arguments are supplied, it is assumed that the grouped object is the first and only input to the applied function. This function returns a dictionary, where keys correspond to each unique group, and values correspond to the function output on each group.
print(tsdframe,"\n")
print(tsdframe.groupby_apply("label", np.mean))
Time (s) a b c
---------- --- ---- -----
0.0 1 2 3
1.0 1 2 3
2.0 1 2 3
3.0 1 2 3
4.0 1 2 3
Metadata
color red blue green
position 10 20 30
... ... ... ...
dtype: int64, shape: (5, 3)
{'x': np.float64(1.5), 'y': np.float64(3.0)}
If the applied function requires additional inputs, these can be passed as additional keyword arguments into groupby_apply.
feature = nap.Tsd(t=np.arange(100), d=np.repeat([0,1], 50))
tsgroup.groupby_apply(
"region",
nap.compute_1d_tuning_curves,
feature=feature,
nb_bins=2)
{'hpc': 3 4
0.25 3.050505 3.919192
0.75 2.969697 4.080808,
'pfc': 1 2
0.25 0.949495 1.898990
0.75 1.070707 2.121212}
Alternatively, an anonymous function can be passed instead that defines additional arguments.
func = lambda x: nap.compute_1d_tuning_curves(x, feature=feature, nb_bins=2)
tsgroup.groupby_apply("region", func)
{'hpc': 3 4
0.25 3.050505 3.919192
0.75 2.969697 4.080808,
'pfc': 1 2
0.25 0.949495 1.898990
0.75 1.070707 2.121212}
An anonymous function can also be used to apply a function where the grouped object is not the first input.
func = lambda x: nap.compute_1d_tuning_curves(
group=tsgroup,
feature=feature,
nb_bins=2,
ep=x)
intervalset.groupby_apply("choice", func)
{'left': 1 2 3 4
0.25 0.847181 1.792115 3.030303 3.551645
0.75 1.077441 2.087542 3.030303 3.838384,
'right': 1 2 3 4
0.25 0.808081 2.154882 2.895623 4.579125
0.75 1.073232 2.398990 2.714646 4.482323}
Alternatively, the optional parameter input_key can be passed to specify which keyword argument the grouped object corresponds to. Other required arguments of the applied function need to be passed as keyword arguments.
intervalset.groupby_apply(
"choice",
nap.compute_1d_tuning_curves,
input_key="ep",
group=tsgroup,
feature=feature,
nb_bins=2)
{'left': 1 2 3 4
0.25 0.847181 1.792115 3.030303 3.551645
0.75 1.077441 2.087542 3.030303 3.838384,
'right': 1 2 3 4
0.25 0.808081 2.154882 2.895623 4.579125
0.75 1.073232 2.398990 2.714646 4.482323}