Engine
In "Engine", you can manage Engines, add external storages, and configure jobs.
If you don't have an Engine yet, you can request one by clicking the Request access now button.
How to access
There are a few ways to access this feature:
Click "Transform your data" on the Studio Home screen
Or go directly to the Data Lake menu
Then, click on "Engine" under the Data Management section.
Once you access the menu, the following screen will be displayed:
Where:
Basic Information
In the first section, you can view the Engine’s name and its description (if available).
Metrics
Monitor the Engine’s performance through the following metrics:
Memory: percentage of memory usage;
Disk: percentage of disk space used;
CPU: percentage of CPU usage. This value may fluctuate;
Total memory: amount of memory allocated;
Total disk: amount of disk space allocated;
CPU cores: number of cores allocated;
View processes: If any processes are running (e.g., SQL), they will be displayed in a list. You can also cancel active processes from this view.
External storage
You can view all external storages registered in the Data Engine. Additionally, you can create, edit, and delete a storage.
Create external storage
To create an external storage, follow these steps:
Click "Create new external storage".
Choose the type of storage and fill in the fields according to the selected type. The types of storage are:
AWS S3
To create a new external storage of type AWS S3, fill in the following fields:
Storage name: Enter a name to identify the storage.
Access Key ID: Insert your AWS Access Key ID.
Secret access key: Insert your AWS Secret Access Key.
Storage region: Select the region where your S3 bucket is located.
Endpoint (optional): Custom endpoint for reading the file within Data Studio.
Storage URL (optional): Custom URL for connecting to S3.
Google Cloud Storage
To create a new external storage of type Google Cloud Storage, fill in the following fields:
Azure
To create a new external storage of type Azure, fill in the following fields:
Storage name: Enter a name to identify the storage.
Service type: Choose the type of service—Blob Storage or Data Lake Storage.
Connection string: Enter the Azure connection string.
Custom Storage
To create a new external storage of type Custom Storage, fill in the following fields:
Storage name: Enter a name to identify the storage.
Access Key ID: Enter the Access Key ID.
Secret access key: Enter the Secret Access Key.
Storage region: Enter the region where the storage is located.
Endpoint: Endpoint used for reading files within Data Studio.
Storage URL: URL of the storage used for authentication.
ℹ️ Tested external storages: localstack and clouds2africa.
To finalize, click "Create".
Edit external storage
To edit external storage, follow these steps:
In the storage list, click "Edit" (pencil icon) on the item you want to modify.
Make the desired changes.
You cannot change the storage type and name.
To finish, click "Save changes".
Delete external storage
To delete external storage, follow these steps:
In the storage list, click "Delete" (trash can icon) on the item you want to remove.
In the modal window that appears, confirm the deletion of the external storage by typing the storage name and then clicking "Delete".
The action cannot be undone.
You can also delete an external storage via the "Edit" option.
Data Consumers
When creating a “file template, you need to configure which users will have access to the endpoint. These users are called "Data Consumers".
Follow the steps below to create, edit, and delete a data consumer:
Create Consumer
To create a data consumer, follow these steps:
Click "Create consumer".
Note that the user will be named user@engine_name. Add the user’s name.
By default, the authentication type is "Basic".
Add a password.
To finish, click "Save user".
Done! The data consumer has been created!
Edit Consumer
To edit a data consumer, follow these steps:
In "Data Consumers", click "Edit" (pencil icon) on the name you want to modify.
Make the desired changes.
You cannot change the user’s name.
To finish, click "Save changes".
Delete Consumer
To delete a data consumer, follow these steps:
In "Data Consumers", click "Delete" (trash can icon) on the name you want to remove.
In the modal window that appears, confirm the deletion of the user by typing the user’s name and then clicking "Delete".
The action cannot be undone.
You can also delete a user via the "Edit" option.
When the data consumer is linked to a file template, the option to Force user deletion will be provided. Forced deletion will remove this link.
ODBC
ODBC (Open Database Connectivity) is a standard for accessing database management systems (DBMS). It allows applications to access different databases in a consistent and standardized manner.
Prerequisites for ODBC Connection
To establish an ODBC connection, ensure that the following requirements are met:
The database information must be available. If the database does not yet exist, it can be created using Skyone Studio.
The user must belong to the same group as the database they wish to access.
The IP used for the connection must be allowed on the Whitelist.
There are two areas for accessing ODBC: Users and Whitelist.
Below, we detail each of them.
Users
The “Users” section allows you to manage connections between the database and Skyone Studio. When accessing this area, you will find:
User Listing: View all registered users.
Search: Use the search bar to find a specific data consumer.
Selection: Select one or more users for simultaneous deletion.
Editing and Deletion: Edit or delete each user individually.
Creating a consumer
Click "Create consumer” and fill in the following fields:
User: Choose the username.
Password: Set a password.
Groups: Select one or more groups. By default, the user has access to the "Default" group.
To finalize, click "Save user”.
The user must belong to the same group as the database they wish to access. Learn more about Groups.
Whitelist
The “Whitelist” section manages access permissions based on IP addresses. When accessing this area, you will find:
Whitelist Listing: View all registered entries, including name, IP address, and status.
Activation/Deactivation: Enable or disable individual whitelist connections.
Editing and Deletion: Edit or delete each whitelist entry independently.
Creating a whitelist
To create a whitelist, fill in the following fields:
Name: Enter the whitelist name.
IP Address: Provide the IP address that will be granted access.
Status: Set the connection status.
To finalize, click "Save”.
The IP address can be found in the error message. Learn more.
Using ODBC with Power BI
Requirements:
Before using ODBC with Power BI, follow these steps:
Install the PostgreSQL driver.
Configure the driver, filling in the required fields.
Steps to connect:
Click “Get Data” in Power BI.
Search for “ODBC”.
Select the previously configured database and click “OK”.
On the first access, enter the password.
Open the Power BI navigator and connect using the appropriate option.
Alternative option: Using SQL
In Power BI, click “Get Data”
Search for “ODBC”.
Select the previously configured database.
Click “Advanced options” and enter the SQL command to retrieve the data.
Click “OK” to view the retrieved information.
Managing IP Connection Errors
1. Connection error due to IP
If a connection error occurs due to an IP restriction, follow these steps:
Check the error message displayed, which will indicate the IP used in the connection attempt.
Copy the IP provided.
Add the IP to the Whitelist to grant access..
Adding the IP to the Whitelist does not automatically enable resources. This action only grants access to the necessary information to establish the connection.
2. How to Find the IP
If you are unsure of the IP address, follow these steps:
Attempt to establish the ODBC connection.
In the error message, locate the IP used in the connection attempt.
Add the identified IP to the Whitelist.
Last updated
Was this helpful?