fix(drilling): drill by pagination works with MSSQL data source

[sfirke]

SUMMARY

fixes #24072 in which Microsoft SQL Server requires an order-by column in order for pagination to succeed. In doing so, now the drilled data is always sorted by the 1st column in ascending order. Previously it retained the dataset's sort order.

Better fixes outside of my ability might be:

• Implement this on a per-database level
• Sort by row number or the equivalent to functionally do nothing and preserve the underlying data order
• Improve user controls for how drilled data displays, allowing the user to control which columns are shown and what the sort order is

But I think the harm introduced (re-sorting the drilled data) is outweighed by fixing this feature for MSSQL.

TESTING INSTRUCTIONS

• CI
• Sam deployed and manually tested the docker image

BEFORE (4.0.0rc1

Drilling.in.4.0.0.take.2-20240313_160744-Meeting.Recording.mp4

AFTER

Post-Patch.MSSQL.Drilling-20240313_160519-Meeting.Recording.mp4

ADDITIONAL INFORMATION

• Has associated issue: The second page of drill-to-detail with MSSQL data source errors"requires an order_by when using an OFFSET or a non-simple LIMIT clause" #24072

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 69.69%. Comparing base (ad9024b) to head (2e580d6).
Report is 1 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #27442      +/-   ##
==========================================
+ Coverage   67.39%   69.69%   +2.30%     
==========================================
  Files        1909     1909              
  Lines       74744    74744              
  Branches     8327     8327              
==========================================
+ Hits        50371    52095    +1724     
+ Misses      22323    20599    -1724     
  Partials     2050     2050              

Flag	Coverage Δ
javascript	57.22% <ø> (ø)
mysql	78.03% <100.00%> (ø)
postgres	78.14% <100.00%> (ø)
presto	53.69% <50.00%> (?)
python	83.06% <100.00%> (+4.77%)	⬆️
sqlite	77.57% <100.00%> (ø)
unit	56.68% <50.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[john-bodley]

@sfirke once this is ready for review would you mind adding the relevant reviewers?

[michael-s-molina]

@sfirke the CI error you're getting

superset/common/query_actions.py:156: error: List item 0 has incompatible type "Tuple[Union[AdhocColumn, str], bool]"; expected "Tuple[Union[AdhocMetric, str], bool]" [list-item]

is because you're passing a Column here

query_obj.orderby = [(query_obj.columns[0], True)]

instead of a Metric which is what OrderBy expects.

Column = Union[AdhocColumn, str]
Metric = Union[AdhocMetric, str]
OrderBy = tuple[Metric, bool]

[sfirke]

I'm stuck, it's unclear to me why this test is failing:

 =================================== FAILURES ===================================
___________ TestPostChartDataApi.test_chart_data_applied_time_extras ___________

self = <tests.integration_tests.charts.data.api_tests.TestPostChartDataApi testMethod=test_chart_data_applied_time_extras>

    @pytest.mark.usefixtures("load_birth_names_dashboard_with_slices")
    def test_chart_data_applied_time_extras(self):
        """
        Chart data API: Test chart data query with applied time extras
        """
        self.query_context_payload["queries"][0]["applied_time_extras"] = {
            "__time_range": "100 years ago : now",
            "__time_origin": "now",
        }
        rv = self.post_assert_metric(CHART_DATA_URI, self.query_context_payload, "data")
>       self.assertEqual(rv.status_code, 200)
E       AssertionError: 400 != 200

And why is test-postgres-presto passing but test-postgres-hive not? Update: Is that test just broken on master branch right now? I see it failed on #27520 😭