Skip to content

Add pipeline operators for <$> and >>= #210

Description

@chfin

Is your feature request related to a problem? Please describe.

When writing pipelines with |>, it is assumed that the dataframe is passed on as is. This prevents the use of monadic operations in pipelines, e.g. operations that can fail and return a Maybe DataFrame or Either Text DataFrame, except as the last step.

Describe the solution you'd like

Add operators that alias functions such as fmap and >>= but with the same precedence and associativity as |>. E.g.:

infixl 9 |>>
(|>>) :: (Functor f) => f a -> (a -> b) -> f b
(|>>) = flip fmap

infixl 9 |>=
(|>=) :: (Monad m) => m a -> (a -> m b) -> m b
(|>=) = (>>=)

This would allow the user to write pipelines such as this:

somePipeline :: DataFrame -> String
somePipeline df =
  df |> op1 -- :: DataFrame
     |> opM -- :: Maybe DataFrame
     |>> op2 -- :: Maybe DataFrame
     |> maybeToEither "some error" -- :: Either String DataFrame
     |>= opE -- :: Either String DataFrame
     |> either id show -- :: String

op1, op2 :: DataFrame -> DataFrame
opM :: DataFrame -> Maybe DataFrame
opE :: DataFrame -> Either String DataFrame

Describe alternatives you've considered

  • Break pipelines where the dataframe enters/leaves a functor/monad and handle transitions manually. This is the current state, but it limits the usability of the pipeline syntax.
  • Leave it to the user to define these operators as needed. Same as above
  • Other names for the operators than suggested above, e.g. |&> instead of |>>, since it's just <&> with a different precedence.

Additional context

Here is a real-world example: The custom operation getKernNotes traverses a column and tries to parse all values, possibly failing with an error message. If successful, the result is added as a new column ("kern"), and several new columns are derived from it. The final step converts the potential error message in the Left case from megaparsec's custom type to Text. Since this operates on the whole Either ... DataFrame value, and not on the dataframe, it can be nicely added to the pipeline using the existing |>.

parseKernTokens df =
  df
    |> filterDataTokens
    |> DT.filterWhere (DT.col @"exclusive" DT..==. DT.lit (Just @T.Text "kern"))
    |> getKernNotes
    |>> DT.derive @"pitch" (DT.lift kernNotePitch $ DT.col @"kern")
    |>> DT.derive @"duration" (DT.lift kernNoteDuration $ DT.col @"kern")
    |>> DT.derive @"tie" (DT.lift kernNoteTie $ DT.col @"kern")
    |>> DT.derive @"grace" (DT.lift kernNoteGrace $ DT.col @"kern")
    |>> DT.derive @"perc" (DT.lift kernNotePerc $ DT.col @"kern")
    |> first (T.pack . errorBundlePretty)
 where
  -- Parse a value, possibly failing with an error message.
  parseToken :: T.Text -> Either (ParseErrorBundle T.Text Void) KernNote
  parseToken = parse parseBasicNote "" 
  -- Parse all values in the "token" column, assign the results to the column "kern".
  getKernNotes df = do
    notes <- traverse parseToken $ DT.columnAsList @"token" df
    pure $ DT.insert @"kern" notes df

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions