Is your feature request related to a problem? Please describe.
When writing pipelines with |>, it is assumed that the dataframe is passed on as is. This prevents the use of monadic operations in pipelines, e.g. operations that can fail and return a Maybe DataFrame or Either Text DataFrame, except as the last step.
Describe the solution you'd like
Add operators that alias functions such as fmap and >>= but with the same precedence and associativity as |>. E.g.:
infixl 9 |>>
(|>>) :: (Functor f) => f a -> (a -> b) -> f b
(|>>) = flip fmap
infixl 9 |>=
(|>=) :: (Monad m) => m a -> (a -> m b) -> m b
(|>=) = (>>=)
This would allow the user to write pipelines such as this:
somePipeline :: DataFrame -> String
somePipeline df =
df |> op1 -- :: DataFrame
|> opM -- :: Maybe DataFrame
|>> op2 -- :: Maybe DataFrame
|> maybeToEither "some error" -- :: Either String DataFrame
|>= opE -- :: Either String DataFrame
|> either id show -- :: String
op1, op2 :: DataFrame -> DataFrame
opM :: DataFrame -> Maybe DataFrame
opE :: DataFrame -> Either String DataFrame
Describe alternatives you've considered
- Break pipelines where the dataframe enters/leaves a functor/monad and handle transitions manually. This is the current state, but it limits the usability of the pipeline syntax.
- Leave it to the user to define these operators as needed. Same as above
- Other names for the operators than suggested above, e.g.
|&> instead of |>>, since it's just <&> with a different precedence.
Additional context
Here is a real-world example: The custom operation getKernNotes traverses a column and tries to parse all values, possibly failing with an error message. If successful, the result is added as a new column ("kern"), and several new columns are derived from it. The final step converts the potential error message in the Left case from megaparsec's custom type to Text. Since this operates on the whole Either ... DataFrame value, and not on the dataframe, it can be nicely added to the pipeline using the existing |>.
parseKernTokens df =
df
|> filterDataTokens
|> DT.filterWhere (DT.col @"exclusive" DT..==. DT.lit (Just @T.Text "kern"))
|> getKernNotes
|>> DT.derive @"pitch" (DT.lift kernNotePitch $ DT.col @"kern")
|>> DT.derive @"duration" (DT.lift kernNoteDuration $ DT.col @"kern")
|>> DT.derive @"tie" (DT.lift kernNoteTie $ DT.col @"kern")
|>> DT.derive @"grace" (DT.lift kernNoteGrace $ DT.col @"kern")
|>> DT.derive @"perc" (DT.lift kernNotePerc $ DT.col @"kern")
|> first (T.pack . errorBundlePretty)
where
-- Parse a value, possibly failing with an error message.
parseToken :: T.Text -> Either (ParseErrorBundle T.Text Void) KernNote
parseToken = parse parseBasicNote ""
-- Parse all values in the "token" column, assign the results to the column "kern".
getKernNotes df = do
notes <- traverse parseToken $ DT.columnAsList @"token" df
pure $ DT.insert @"kern" notes df
Is your feature request related to a problem? Please describe.
When writing pipelines with
|>, it is assumed that the dataframe is passed on as is. This prevents the use of monadic operations in pipelines, e.g. operations that can fail and return aMaybe DataFrameorEither Text DataFrame, except as the last step.Describe the solution you'd like
Add operators that alias functions such as
fmapand>>=but with the same precedence and associativity as|>. E.g.:This would allow the user to write pipelines such as this:
Describe alternatives you've considered
|&>instead of|>>, since it's just<&>with a different precedence.Additional context
Here is a real-world example: The custom operation
getKernNotestraverses a column and tries to parse all values, possibly failing with an error message. If successful, the result is added as a new column ("kern"), and several new columns are derived from it. The final step converts the potential error message in theLeftcase from megaparsec's custom type toText. Since this operates on the wholeEither ... DataFramevalue, and not on the dataframe, it can be nicely added to the pipeline using the existing|>.