Skip to main content
mkohorst
Member
September 15, 2026
Solved

DataFrame Question

  • September 15, 2026
  • 1 reply
  • 86 views

DataFrame question: Today we use DataTables and LINQ to perform operations such as multi-column GroupBy, aggregations (Sum, Count, Average), joins, filtering, and calculated columns. Do DataFrames support these types of data-shaping operations natively, and if so, are they generally more efficient than DataTable + LINQ for in-memory transformations? For example, replacing a multi-column GroupBy + Sum pattern like the one below: 

// Group by Entity, ProductType, ProductLine, Account, Channel, sum(LOCALAMOUNT)
DataTable result = dt.AsEnumerable()
    .GroupBy(r => new
    {
        ENTITY = r.Field<string>("ENTITY"),
        PRODUCTTYPE = r.Field<string>("PRODUCTTYPE"),
        PRODUCTLINE = r.Field<string>("PRODUCTLINE"),
        ACCOUNT = r.Field<string>("ACCOUNT"),
        CHANNEL = r.Field<string>("CHANNEL")
    })
    .Select(g =>
    {
        DataRow row = dt.NewRow();
        row["ENTITY"] = g.Key.ENTITY;
        row["PRODUCTTYPE"] = g.Key.PRODUCTTYPE;
        row["PRODUCTLINE"] = g.Key.PRODUCTLINE;
        row["ACCOUNT"] = g.Key.ACCOUNT;
        row["CHANNEL"] = g.Key.CHANNEL;
        row["Amount"] = g.Sum(r => Convert.ToDecimal(r["LOCALAMOUNT"]));
        return row;
    })
    .CopyToDataTable();

return result;

      Best answer by JackLacava

       

      Transcript

      So we had a great question from MKohorst about DataFrames. And they use data tables today to do advanced operations like groupbys some, some counts and things like that. And they're asking, can I do the same things with DataFrames. Do they support the same stuff?

      DataFrames are focused on being lightweight and performant, so they don't necessarily have all the features that were added to data tables by Microsoft over the last 25 years. Right. But that's precisely why we added a super simple method called to data table on DataFrames, so that if you really need a data table, you can have it basically no cost you. You create a DataFrame first and then you convert it to a data table when you need it.

      Whenever the initial construct is retrieving records from an existing table, chances are that it will still be significantly faster to perform the initial query throughout DataFrame, and then convert it to a data table for the advanced operations, then just sticking to data table all the way through. So I would suggest try to do the sort of the first retrieval anyway in DataFrames and then convert to data table if you really have a need for a for that, for that sort of advanced stuff. This said, DataFrames do have a couple of methods for very common operations, like for each row to work on specific fields faster than retrieving the whole row.

      And we are still looking at adding more facilities going forward, but there's always going to be something that DataFrames cannot do compared to a data table, because they are different things for different situations. So so yeah, but yeah, thanks for the question. That was a great question.

      1 reply

      Sage
      JackLacavaSageAnswer
      Sage
      September 22, 2026

       

      Transcript

      So we had a great question from MKohorst about DataFrames. And they use data tables today to do advanced operations like groupbys some, some counts and things like that. And they're asking, can I do the same things with DataFrames. Do they support the same stuff?

      DataFrames are focused on being lightweight and performant, so they don't necessarily have all the features that were added to data tables by Microsoft over the last 25 years. Right. But that's precisely why we added a super simple method called to data table on DataFrames, so that if you really need a data table, you can have it basically no cost you. You create a DataFrame first and then you convert it to a data table when you need it.

      Whenever the initial construct is retrieving records from an existing table, chances are that it will still be significantly faster to perform the initial query throughout DataFrame, and then convert it to a data table for the advanced operations, then just sticking to data table all the way through. So I would suggest try to do the sort of the first retrieval anyway in DataFrames and then convert to data table if you really have a need for a for that, for that sort of advanced stuff. This said, DataFrames do have a couple of methods for very common operations, like for each row to work on specific fields faster than retrieving the whole row.

      And we are still looking at adding more facilities going forward, but there's always going to be something that DataFrames cannot do compared to a data table, because they are different things for different situations. So so yeah, but yeah, thanks for the question. That was a great question.

      Terms & ConditionsAccessibility statement