Server refresh not working after migration to 2020.4.2 #444

lgemeinhardt · 2021-03-08T07:50:17Z

After upgrade (from 2020.3.3 to 2020.4.2) the web data connector could not refresh anymore (Error message: Unknown Failure ( status code = 3002, The connection to the data source might have been lost. ExternalProtocol::PipeMessageSource::ReadBytes: Communication with the Tableau Protocol Server process was lost. Tableau Protocol Server process might be terminated, possibly due to system resource constraints or another external source ) ... same wdc was running perfect in previous versions and runs perfect in desktop tool (even with version 2020.4.2). Whitelist is not the issue and I could see the request also on the server site (storing the wdc). Also a fresh sample like the one below works only in desktop, but not with server refresh anymore - look like the server does not call getSchema or getData callback

<html>
<head>
    <title>Basic WDC</title>
    <meta http-equiv="Cache-Control" content="no-store" />
    <script src="https://connectors.tableau.com/libs/tableauwdc-2.3.latest.js" type="text/javascript"></script>
</head>
<body>
    <script type="text/javascript">
		var myConnector = tableau.makeConnector(); 
		myConnector.getSchema = function(schemaCallback) {
			var cols = [{
				id: "id",
				dataType: tableau.dataTypeEnum.string
			}, {
				id: "title",
				alias: "title",
				dataType: tableau.dataTypeEnum.string
			}];
			var tableSchema = {
				id: "basic",
				columns: cols
			};
			schemaCallback([tableSchema]);
		};
		myConnector.getData = function(table, doneCallback) {
			var tableData = [];
			tableData.push({
				"id": "1",
				"title": "title"
			});
			table.appendRows(tableData);
			doneCallback();
		};
		tableau.registerConnector(myConnector);
		function runReport() {
			tableau.connectionName = "Basic";
			tableau.submit();
		}
	</script>
	<button id="submit" onclick="runReport()">Run</button>
</body>
</html>

The text was updated successfully, but these errors were encountered:

another-tom · 2021-04-15T15:36:23Z

We have run into this issue with other WDCs' after upgrading to 2020.4. Based on this post I was able to apply the below changes and get them working.
tsm configuration set -k features.WDC_QtWebEnginePort -v false

According to Tableau Support.

QtWebEngine has been upgraded from 5.10 to 5.15 between the two versions of Tableau Server (2020.2.2 and 2020.4.3) which may affect WDCs in unexpected ways. 5.10 was based on Chromium 60 and 5.15 is based on Chromium 80 which had significant security changes.

As with any Tableau Server upgrade, we always recommend a comprehensive testing environment to help mitigate issues such as these. We still highly recommend that the WDC author review the two resources previously provided.

Tableau provides limited customer support for connections using the Web Data Connector. Tableau does not provide support for connectors or for other programs written to interface with the WDC API. However, you can submit questions and ask for help on the Tableau developer community forums.

Tableau does provide support for the WDC library and SDK though. If you find an issue with the WDC library, the simulator, or any of the developer samples, please submit an issue on Github.

If the WDC was not developed by someone on your team, we would recommend reaching out to the author so they may look into the above resources for support with their WDC.
As QtWebKit is considered a legacy component and is currently still in Tableau Server for compatibility, we cannot guarantee that it will not be eventually removed from the codebase at a later date. So far, I see no near-term plans for that, but development shifts over time.

Tom S.
Technical Support

lgemeinhardt · 2021-04-16T10:05:10Z

@tomslickcook Thank you very much for the hint!
This enables my WDC again, even if I see it as workaround (using a legacy component) and not a fix.
In addition I have no glue what security could be in my very basic WDC above, because it contains only content from the samples pages (and I assume they should work).

cshort727 · 2021-04-16T13:31:14Z

We're having the same issue after upgrading to Tableau Server 2021.1, none of our WDCs will refresh on the Server. I tried the above workaround but it didn't work and then I was told by Tableau Support that unfortunately that workaround does not apply to 2021.1 and forward but will work for 2020.4 for now.

mellolr1 · 2021-04-19T13:14:55Z

We also upgraded from from 2020.3.3 to 2020.4.2, and I'm having the same error as @lgemeinhardt with our custom WDC. The change suggested by @tomslickcook was applied, but the WDC still isn't working.

Looking at a trace from IIS, the Tableau server is not negotiating authentication, it is only trying anonymous and failing.

Using Chromium 92.0 I am able to run the WDC from the desktop. I am also able to refresh the extract using Tableau desktop.

lgemeinhardt · 2021-04-19T13:43:12Z

@mellolr1 Strange - did you run "tsm pending-changes apply" to force the server to load the changes?

mellolr1 · 2021-04-19T20:02:05Z

Yes, we did. I'm seeing this error in the dashboard.
com.tableausoftware.nativeapi.exceptions.DataSourceException: SyntaxError: Expected an identifier but found 'cachedTableData' instead file: ....js line: 1
line 1 in that JS is let cachedTableData; So I changed it to var cachedTableData;
Now the error is in _retrieveJsonData({ dataUrl, method }, function (jsonData) { SyntaxError: Unexpected token ','
This used to work prior to 2020.4.

lgemeinhardt · 2021-04-20T06:21:01Z

@mellolr1 Looks for me like a different issue, because I cannot refresh the extract, but could use an extract populated via Tableau Desktop in the dashboard ... I assume your issue is more related to the dashboard.

mellolr1 · 2021-04-20T10:31:55Z

Those erros only happen in the server. I am able to refresh the extracts using this WDC from the desktop.

KeshiaRose · 2021-04-20T15:24:59Z

@mellolr1 are you using the json/xml connector? Once you switch the WebEngine flag off on Server you no longer have access to the improvements that came with it, like being able to use modern javascript (ie: const/let, arrow functions, etc.) Therefore that WDC won't work anymore on Server, but on Desktop, which is still using the current WebEngine, it will work fine.

mellolr1 · 2021-04-20T18:37:43Z

@KeshiaRose yes that's what I started with. Is that the old WebKit vs new WebEngine? Is there a way to test my JS with WebKit without pushing to the server?

mellolr1 · 2021-04-21T12:13:13Z

I think I got it now, thank you @KeshiaRose for putting me on the right track. I installed desktop 2019.3 to troubleshoot the WDC without WebEngine. What a mess... but almost back in business.

mellolr1 · 2021-05-19T17:29:19Z

Any news or updates on this issue or is disabling WDC_QtWebEnginePort still the only workaround?

With @KeshiaRose 's help I got our connector working when QtWebEngine is disabled, but our new environment will have it enabled again.
I see the requests being made to our webserver. From the desktop it works. I see it requesting the html, then the js and lastly it makes the call to REST using BASIC authentication. From the server it keeps requesting the html, then the js in a loop until it times out. The error on the server after 117 secs:

"java.lang.RuntimeException: The connection to the data source might have been lost.
ExternalProtocol::PipeMessageSource::ReadBytes: Communication with the Tableau Protocol Server process was lost. Tableau Protocol Server process might be terminated, possibly due to system resource constraints or another external source"

This is the js connector...

var cachedTableData;
var
    dataUrl = '/RProxy/****',
    method = 'get',
    token,
    cols = [],
    tableSchema = {
        id: "DataID",
        alias: "Data ID"
    };
var wdConnector = tableau.makeConnector();

wdConnector.getSchema = function (schemaCallback) {
    _retrieveJsonData(dataUrl, method, function (jsonData) {
        var data = jsonData.Data.records;
        var rawFields = Object.keys(data[0]);
        var cols = [];
        for (var i = 0, len = rawFields.length; i < len; i++) {
            var dataField = rawFields[i];
            var obj = {};
            obj.id = dataField.replace(/\$/g, 'attr').replace(/[^A-Za-z0-9_]/g, '_');
            obj.alias = _getAlias(obj.id);
            if (obj.id.toLowerCase().indexOf('date') != -1) {
                obj.dataType = tableau.dataTypeEnum.date;
            }
            else {
                obj.dataType = tableau.dataTypeEnum.string;
            }
            cols.push(obj);
        }
        tableSchema.columns = cols;
        console.log('Table schema created: ', tableSchema);
        schemaCallback([tableSchema]);
    });
};
wdConnector.getData = function (table, doneCallback) {
    _retrieveJsonData(dataUrl, method, function (rawData) {
        var
            tableData = [],
            data = rawData.Data.records,
            rawFields = Object.keys(data[0]);
        // Iterate over the rawData
        for (var i = 0; i < data.length; i++) {
            var obj = {};
            for (var x = 0; x < rawFields.length; x++) {
                var dataField = rawFields[x];
                var id = dataField.replace(/\$/g, 'attr').replace(/[^A-Za-z0-9_]/g, '_');
                obj[id] = data[i][dataField];
            }
            tableData.push(obj);
            tableau.reportProgress("Getting row: " + i);
        }
        table.appendRows(tableData);
        doneCallback();
    });
};
wdConnector.init = function (initCallback) {
    tableau.authType = tableau.authTypeEnum.basic;
    initCallback();
}
tableau.registerConnector(wdConnector);

// Gets data
function _retrieveJsonData(dataUrl, method, retrieveDataCallback) {
    var rawData = '';
    var username = tableau.username;
    var password = tableau.password;

    var successCallback = function (data) {
        cachedTableData = JSON.parse(data);
        retrieveDataCallback(cachedTableData);
    };

    var ajaxCallback = function (rawData) {
        try {
            successCallback(rawData);
        }
        catch (e) {
            console.log(rawData, dataUrl, e);
            tableau.abortWithError(rawData);
        }
    };

    if (!cachedTableData) {
        $.ajax({
            url: dataUrl,
            type: method,
            timeout: 25000,
            // Add basic authentication headers to your request like this. Note that
            // the password is encrypted when stored by Tableau; the username is not.
            headers: {
                "Authorization": "Basic " + btoa(username + ":" + password),
                "accept":"application/json"
            },
            dataType: "text",
            success: ajaxCallback,
            // Use this.ajaxErrorHandler for basic error handling.
            error: function (jqXHR, textStatus, errorThrown) {
                console.error("ajax error: ", jqXHR, textStatus, errorThrown);
                _error(errorThrown);
                return false;
            }
        });
    } else {
        retrieveDataCallback(cachedTableData);
        return;
    }
}
function _getAlias(str) {
    var parts = str.split('_');
    var alias = '';
    for (var i = 0; i < parts.length; i++) {
        if (parts[i].trim().substring(0,1)==='x') {
            alias += String.fromCharCode('0' + parts[i]);
        }
        else {
            alias += parts[i];
        }
    }
    return alias;
}
// Shows error message below submit button
function _error(message) {
    $('.error').fadeIn('fast').delay(3000);
    $('.error').html(message);
    $('html, body').animate({ scrollTop: $(document).height() }, 'fast');
    try {
        if (tableau.phase !== 'interactive') {
            console.error(result.error);
            tableau.abortWithError(result.error);
        } else {
            tableau.abortWithError(message);
        }
    } catch (e) { console.error('_error', e); }
}
function _chechAuth(username,password) {
    $.ajax({
        url: '/RProxy/myip',
        type: 'get',
        timeout: 1000,
        // Add basic authentication headers to your request like this. Note that
        // the password is encrypted when stored by Tableau; the username is not.
        headers: {
            "Authorization": "Basic " + btoa(username + ":" + password)
        },
        dataType: "text",
        success: function () {
            tableau.connectionName = "HealthRx_WebService";
            tableau.username = username;
            tableau.password = password;
            tableau.submit();
            return false;
        },
        // Use this.ajaxErrorHandler for basic error handling.
        error: function (jqXHR, textStatus, errorThrown) {
            console.error("_chechAuth error: ", jqXHR, textStatus, errorThrown);
            $('.error').fadeIn('fast').delay(3000);
            $('.error').html('Authentication failed for: '+username);
            $('html, body').animate({ scrollTop: $(document).height() }, 'fast');
            $('#username').val('');
            $('#password').val('');
            return false;
        }
    });
}
$(document).ready(function () {
    $('#datasource').html('<h3>Data Source: ' + tableSchema.alias + '</h3');

    $("#submitButton").click(function () {
        $('.error').hide();
        var username = $('#username').val();
        var password = $('#password').val();
        _chechAuth(username, password);
    });
});

`

lgemeinhardt · 2021-06-24T09:54:15Z

Link this with https://community.tableau.com/s/question/0D54T00000WVtT2/since-upgrade-to-202042-any-workbooks-with-a-wdc-datasource-when-extract-refreshes-on-server-cpus-max-out-at-100 ... Issue still not solved 👎

KeshiaRose · 2021-07-27T18:53:33Z

Looks like this has been resolved, here are the details:

"Our support team has identified the issue with the newer QTWebEngine.

The older version of QTWebEngine could run in an elevated mode in Windows without issue. The new version, for security reasons, will not run in elevated mode.

Starting in 2021.1, the legacy version of QTWebEngine was not longer shipped with the product, so the legacy mode was no longer available. For this reason, users running Tableau Server on Windows will need to read and perform the steps below.

We have resolved this issue for multiple customers by removing the Run As User (RAU) from the local administrator group on the Windows Server. In some cases, the RAU was also a member of other groups which were added to the local administrator group. Making sure that the Run As User is not in any of the local administrator groups will prevent the child process for QTWebEngine from running in an elevated state and allow it to function correctly.

It becomes extremely important to make sure that the RAU has the correct settings as documented here:
https://help.tableau.com/current/server/en-us/runas_confirm.htm"

I'm going to close this issue but please re-open if after trying these steps you are still seeing the problem.

Note: If you had set the WDC_QtWebEnginePort flag to false as a remedy I recommend returning it back to true as this will allow you to use WDCs that have modern JavaScript (tsm configuration set -k features.WDC_QtWebEnginePort -v true).

theoutlander assigned maxrandolph Jun 3, 2021

KeshiaRose closed this as completed Jul 27, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Server refresh not working after migration to 2020.4.2 #444

Server refresh not working after migration to 2020.4.2 #444

lgemeinhardt commented Mar 8, 2021

another-tom commented Apr 15, 2021

lgemeinhardt commented Apr 16, 2021

cshort727 commented Apr 16, 2021

mellolr1 commented Apr 19, 2021

lgemeinhardt commented Apr 19, 2021

mellolr1 commented Apr 19, 2021 •

edited

Loading

lgemeinhardt commented Apr 20, 2021

mellolr1 commented Apr 20, 2021

KeshiaRose commented Apr 20, 2021

mellolr1 commented Apr 20, 2021

mellolr1 commented Apr 21, 2021

mellolr1 commented May 19, 2021

lgemeinhardt commented Jun 24, 2021

KeshiaRose commented Jul 27, 2021

Server refresh not working after migration to 2020.4.2 #444

Server refresh not working after migration to 2020.4.2 #444

Comments

lgemeinhardt commented Mar 8, 2021

another-tom commented Apr 15, 2021

lgemeinhardt commented Apr 16, 2021

cshort727 commented Apr 16, 2021

mellolr1 commented Apr 19, 2021

lgemeinhardt commented Apr 19, 2021

mellolr1 commented Apr 19, 2021 • edited Loading

lgemeinhardt commented Apr 20, 2021

mellolr1 commented Apr 20, 2021

KeshiaRose commented Apr 20, 2021

mellolr1 commented Apr 20, 2021

mellolr1 commented Apr 21, 2021

mellolr1 commented May 19, 2021

lgemeinhardt commented Jun 24, 2021

KeshiaRose commented Jul 27, 2021

mellolr1 commented Apr 19, 2021 •

edited

Loading