Book a Demo!
CoCalc Logo Icon
StoreFeaturesDocsShareSupportNewsAboutPoliciesSign UpSign In
pola-rs
GitHub Repository: pola-rs/polars
Path: blob/main/crates/polars-lazy/src/dsl/mod.rs
6939 views
1
//! Domain specific language for the Lazy API.
2
//!
3
//! This DSL revolves around the [`Expr`] type, which represents an abstract
4
//! operation on a DataFrame, such as mapping over a column, filtering, group_by, or aggregation.
5
//! In general, functions on [`LazyFrame`]s consume the [`LazyFrame`] and produce a new [`LazyFrame`] representing
6
//! the result of applying the function and passed expressions to the consumed LazyFrame.
7
//! At runtime, when [`LazyFrame::collect`](crate::frame::LazyFrame::collect) is called, the expressions that comprise
8
//! the [`LazyFrame`]'s logical plan are materialized on the actual underlying Series.
9
//! For instance, `let expr = col("x").pow(lit(2)).alias("x2");` would produce an expression representing the abstract
10
//! operation of squaring the column `"x"` and naming the resulting column `"x2"`, and to apply this operation to a
11
//! [`LazyFrame`], you'd use `let lazy_df = lazy_df.with_column(expr);`.
12
//! (Of course, a column named `"x"` must either exist in the original DataFrame or be produced by one of the preceding
13
//! operations on the [`LazyFrame`].)
14
//!
15
//! [`LazyFrame`]: crate::frame::LazyFrame
16
//!
17
//! There are many, many free functions that this module exports that produce an [`Expr`] from scratch; [`col`] and
18
//! [`lit`] are two examples.
19
//! Expressions also have several methods, such as [`pow`](`Expr::pow`) and [`alias`](`Expr::alias`), that consume them
20
//! and produce a new expression.
21
//!
22
//! Several expressions are only available when the necessary feature is enabled.
23
//! Examples of features that unlock specialized expression include `string`, `temporal`, and `dtype-categorical`.
24
//! These specialized expressions provide implementations of functions that you'd otherwise have to implement by hand.
25
//!
26
//! Because of how abstract and flexible the [`Expr`] type is, care must be take to ensure you only attempt to perform
27
//! sensible operations with them.
28
//! For instance, as mentioned above, you have to make sure any columns you reference already exist in the LazyFrame.
29
//! Furthermore, there is nothing stopping you from calling, for example, [`any`](`Expr::any`) with an expression
30
//! that will yield an `f64` column (instead of `bool`), or `col("string") - col("f64")`, which would attempt
31
//! to subtract an `f64` Series from a `string` Series.
32
//! These kinds of invalid operations will only yield an error at runtime, when
33
//! [`collect`](crate::frame::LazyFrame::collect) is called on the [`LazyFrame`].
34
35
pub mod functions;
36
37
pub use functions::*;
38
pub use polars_plan::dsl::*;
39
pub use polars_plan::plans::UdfSchema;
40
41