FiftyOne dataset from LakeFS - Advanced¶
Category: data_generator.vision.detection.2d
Version: 1.0.0
UUID: e9fa416ae22b851219989365dd7be45360dbda4b2a15df61af785fa94de81865
Execution context: leip_af
Choice Priority: 1.0
Subcomponent Parameters¶
| Name | Synonyms | Allowable Categories |
|---|---|---|
| Training Augmentation | data.augmentation.training |
augmentation.vision.2d.training |
| Validation Augmentation | data.augmentation.validation |
augmentation.vision.2d.validation |
Value Parameters¶
| Name | Synonyms | Type | Values | Help |
|---|---|---|---|---|
| Number of files per batch | data.lakefs_batch_size |
scalar | int | Number of files to process in each batch when downloading from LakeFS |
| Download chunk size (in bytes) | data.lakefs_chunk_size |
scalar | int | Size of each download chunk in bytes when fetching data from LakeFS |
| LakeFS commit ID that uniquely identifies the dataset version | data.lakefs_commit_id |
scalar | string | Specify the commit ID in LakeFS that corresponds to the dataset version you want to use. This is the ultimate reproducibility identifier for the dataset in LakeFS. |
| Fiftyone Dataset Type | data.fiftyone_dataset_type |
scalar | string | Specify the type of dataset in FiftyOne, e.g., 'detection', 'classification', etc. |
| Fiftyone Groundtruth Field Name | data.fiftyone_groundtruth_field_name |
scalar | string | |
| Label | data.fiftyone_label_map |
dict | int | Make sure the label_map is in str2idx format eg. |
| Maximum concurrent downloads | data.lakefs_max_concurrent |
scalar | int | Maximum number of concurrent downloads when fetching data from LakeFS |
| LakeFS prefix | data.lakefs_prefix |
scalar | string | |
| LakeFS repository name | data.lakefs_repo |
scalar | string | |
| Use concurrent downloads | data.lakefs_use_concurrent |
scalar | bool | Whether to use concurrent downloads when fetching data from LakeFS |
| Validate local data integrity | data.lakefs_validate_integrity |
scalar | bool | Whether to validate local data integrity, if False will skip download if folder exists locally but wont verify validity. |
| Fiftyone Train View Name | data.fiftyone_train_view |
scalar | string | |
| Fiftyone Val View Name | data.fiftyone_val_view |
scalar | string | |
| Number of classes | data.number_of_classes |
scalar | int |
Constraints¶
- Number of classes has to be positive (meta.nclasses > 0)
- LakeFS repository name must be provided (actual.repo != '')
- LakeFS commit ID must be provided (actual.commit_id != '')
- LakeFS prefix must be provided (actual.prefix != '')