Quick Dataframes for Huge Issues
I used to be just lately in a secret demo run by the Cuda and Polars workforce. They handed me by means of a metallic detector, put a bag over my head, and drove me to a shack within the woods of rural France. They took my telephone, pockets, and passport to make sure I wouldn’t spill the beans earlier than lastly exhibiting off what they’ve been engaged on.
Or, that’s what it felt like. In actuality it was a zoom assembly the place they politely requested me to not say something till a specified time, however as a tech author the thriller had me feeling just a little like James Bond.
On this article we’ll focus on the content material of that assembly: a brand new execution engine in Polars that allows GPU accelerated computation, permitting for interactive information manipulation of 100GB+ of information. We’ll focus on what an information body is in polars, how GPU acceleration works with polars dataframes, and the way a lot of a lift to efficiency one can count on with the brand new CUDA powered execution engine.
Who’s this convenient for? Anybody who works with information and desires to work quicker.
How superior is that this put up? This put up accommodates easy however cutting-edge information engineering ideas. It’s related to readers of all ranges.