[SYSTEMDS-XXXX] Add coresetDT builtin for coreset selection using a decision tree#2540
Open
Kenny-Dean wants to merge 1 commit into
Open
[SYSTEMDS-XXXX] Add coresetDT builtin for coreset selection using a decision tree#2540Kenny-Dean wants to merge 1 commit into
Kenny-Dean wants to merge 1 commit into
Conversation
…ecision tree This patch includes coresetDT, a builtin that selects a smaller training coreset from a decision-tree datamap, with a test on the wine dataset.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The method is adapted from the datamap-driven coreset approach of Hadar et al.,
"Datamap-Driven Tabular Coreset Selection for Classifier Training"
(https://www.vldb.org/pvldb/vol18/p876-razmadze.pdf).
This patch includes:
The implementation works as intended, datasets are compressible down to a fraction of the full dataset while keeping the full datasets accuracy, however, parameter tweaking of frac, psi, tau, samp_ratio and max tree depth is possible, which allows someone to shift the focus from model hyperparameter tuning to coreset optimization.
The current default parameters are results from some sweeps over the local Titanic and Wine dataset as well as the Adult and a subsampled version on Covertype dataset, which all are medium in row size and manageablein coloum size.
Deviations from the default parameters of the paper include the psi (1 -> 0.9), tau (5 -> 10), datamap model (GBDT -> 1DT)(variation explained in paper), whereas samp_ratio stayed the same. This produces good results.
Some statistics:
Compressability and the floor are different for each dataset, so its advised to first run the function with verbose=True to get some insights over the region statistics and the dataset- (and parameter-) specific coreset floor.
Future work might include the implementation of the opt_per algorithm that is described in the same paper. It returns the minimal coreset given an accuracy drop you are willing to accept, but the ability to perform parameter sweeps for the best parameters is still possible.
PS: Since this Student Project was given to me without a Jira Issue present the title is only a placeholder for now.