Troubleshooting: Resolving query execution errors in the Data Warehouse

Running excessively heavy queries can lead to processing container crashes. As a reactive measure, the system performs up to three automatic execution attempts. If the failure persists, the process is terminated to protect the environment's integrity.

This article guides you through investigating these errors and applying optimization techniques to stabilize your DataJobs.

Error Signs

The error is identified in two ways:

• In the Skyone Studio: The error message “Multiple attempts were made to process the query. Optimize the query and run it again” will appear directly in the interface, within the Output tab of the Data Warehouse.

• In the Workflow: The user will receive an automatic email reporting the execution failure.

What does this mean?

This message indicates that the system made three consecutive attempts to execute the query, but all failed. Usually, this occurs because the query is excessively heavy (e.g., processing hundreds of thousands of rows), causing the execution container to crash and automatically reboot.

Investigation and Diagnosis

To resolve the issue, you must identify whether the failed query is the root cause or a collateral victim. Check the following points:

1. Logic Review

The first step is to review the query logic and verify if the data volume exceeds the environment's capacity.

2. Parallel execution issues

Multiple queries may run simultaneously in the same environment. A single "guilty" query can consume all resources (Memory/CPU), crashing the container and triggering errors for all other queries running at that time.

• Action: Analyze what was running concurrently to isolate the query that caused the failure.

Best practices for prevention and correction

Follow these optimization recommendations to avoid crashes:

Query optimization (SQL)

• Using LIMIT: When performing tests or selections on large volumes, use the LIMIT clause to restrict the initial data load.

• Column refactoring: Avoid using SELECT *. Specify only the necessary columns to reduce the processing load.

Memory settings and DuckDB

• Memory management: Ensure the query is correctly sized for the container's memory limits.

• Parameter adjustments: Consider adding specific DuckDB configurations to improve stability and performance in complex operations.

Read more: DuckDB Documentation

Practical adjustment example

Error scenario

Below is an example of a query that may cause resource exhaustion when attempting to load massive data without filters:

-- Unoptimized query: high risk of crashing
SELECT * FROM test_db.giant_table;

Recommended adjustment

By applying filters and limiting the scope, the risk of a container crash is minimized:

-- Optimized query: focused on performance and stability
SELECT id, name, processing_date 
FROM test_db.giant_table 
WHERE processing_date >= '2026-01-01'
LIMIT 1000;

If the error persists even after optimization, conduct a separate troubleshooting investigation to check parallel activity logs and overall environment health.