Sunday, December 8, 2024

ETL TESTING TRAINING - notes

 

 

 

STTM 

 

 

 

 

TEST CASES  

A screenshot of a spreadsheet

Description automatically generated 

A computer screen shot of a computer program

Description automatically generated 

 

A close-up of a white background

Description automatically generated 

A screenshot of a computer

Description automatically generated 

STTM – TESTING 

  1. Whether same datatype in source and target or not 

  1. A close-up of a white screen

Description automatically generated 

  1. A screenshot of a computer code

Description automatically generated 

  1. A screenshot of a computer code

Description automatically generated 

  1. A computer screen shot of a code

Description automatically generated 

 

A computer code with black text

Description automatically generated 

 

 

 

A computer screen shot of a computer code

Description automatically generated 

The value should be both zero , viceversa 

 

Incremental data validation 

Already loaded data , if we run again on that data is called incremental like SCD functionalities . 

 

A white screen with black text

Description automatically generated 

 

A white background with black text

Description automatically generated 

Means if there are any changes in the CUSTOMER table the old functionalities should not be impacted and new functionalities shd be tested as well  

 

 

 

 

 

 

 

 

 

 

  

 

 

 

 

 

 

Active : only load dept =10 records , num of records reduce 

Passive : name = fname + lnam , num of records don’t decrease  

 

 

 

 

 

 

 

 

 

 

 

 

AGILE METHODOLOGY 

 

Sprint :  

  • Sprint planning ( First  day of sprint ) 

  • Sprint story points ( jira )  

  • Sprint daily call  

  • Sprint retrospective ( Last day of sprint ) 

  • Sprint backlog ( planned , Actual , moved to backlog )  

  •  

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

RESUME CONTENT 

 

 

A close-up of a computer screen

Description automatically generated 



ETL testing notes / Data modeling / SQL - index



  1. My Doubts

  1. Data Modeling Doubuts 

  1. How the size is calculated for each column with a datatype 

  2. How to see the performance of a Table in a Data model 

  3. How to optimize the performance of a Table in a Data model

  1. Performance Tuning Doubts

  2. Requirement Gathering questions 

Sfs 

  1. Dsfds

  1. SQL

  1. COPY ROWS FROM ONE TABLE TO ANOTHER 

 

USE db1;

SHOW tables;

SELECT * FROM   emp;

DROP TABLE emp3;

CREATE TABLE emp3 LIKE emp;

SELECT *FROM   emp3;

INSERT INTO emp3 SELECT * FROM   emp WHERE  id = 1; 

  1. How MYSQL autocommit off 


  •   set autocommit =0;

  •   

  •  INSERT INTO emp3 SELECT * FROM   emp WHERE  id = 1; 

  •   select * from emp3;


  •   rollback;

  1. Create user and grant privileges

  • Create the User: Replace newuser with your desired username and password with the desired password.

  • sql

  • CREATE USER 'newuser'@'localhost' IDENTIFIED BY 'password';

  • Grant Privileges: Assign the necessary privileges to the new user. For example, to grant all privileges on a specific database:

  • sql

  • GRANT ALL PRIVILEGES ON your_database.* TO 'newuser'@'localhost';

  • Flush Privileges: Apply the changes.

  • sql

  • FLUSH PRIVILEGES;

  • Exit MySQL:

  • sql

  • EXIT;

  • Now, newuser can log 

  • Change User Password

  • To change the password for an existing user:

  • sql

  • ALTER USER 'username'@'host' IDENTIFIED BY 'new_password';

  • Or for MySQL 5.7.6 and newer:

  • sql

  • SET PASSWORD FOR 'username'@'host' = 'new_password';

  • Grant Additional Privileges

  • To grant new privileges to an existing user:

  • sql

  • GRANT ALL PRIVILEGES ON database_name.* TO 'username'@'host';

  • Or a more specific privilege:

  • sql

  • GRANT SELECT, INSERT ON database_name.* TO 'username'@'host';

  • Rename a User

  • To rename an existing user (MySQL 5.7.8+):

  • sql

  • RENAME USER 'old_username'@'host' TO 'new_username'@'host';

  • Revoke Privileges

  • To revoke privileges from a user:

  • sql

  • REVOKE ALL PRIVILEGES, GRANT OPTION FROM 'username'@'host';

  • Drop a User

  • To delete an existing user:

  • sql

  • DROP USER 'username'@'host';

  • Remember to always flush privileges after making changes to ensure they take effect:

  • sql

  • FLUSH PRIVILEGES;

  • If you have any specific changes in mind, let me know, and I can guide you through it!



  1. FLUSH AND REVOKE COMMANDS

  • FLUSH

  • Purpose: The FLUSH command is used to reload, reset, or clear various caches and logs. When you FLUSH PRIVILEGES, it reloads the privilege tables, making any changes to user accounts or permissions effective immediately.

  • Example:

  • sql

  • FLUSH PRIVILEGES;

  • REVOKE

  • Purpose: The REVOKE command is used to remove specific privileges from a user. It explicitly takes away permissions that have been granted to a user.

  • Example:

  • sql

  • REVOKE ALL PRIVILEGES, GRANT OPTION FROM 'username'@'host'



Friday, November 22, 2024

Different data modeling tools list

 

===========================================
AIR TABLE : (3jul 2025)
Data modeling tool with AI data options and self creating data model 


===========================================


===========================================
ToolCreatorTarget Business SizeLicenseSupported Database PlatformsSupported OSsStandalone or bundled into a larger toolkitLaunch Date
AstahChange VisionEnterprisesProprietaryMySQLOracle,Windows, macOS, LinuxStandalone2006
Database WorkbenchUpscene ProductionsSMBs and enterprisesProprietaryMS SQL ServerMySQLOracleFirebirdInterBaseSQL AnywhereNexusDBMariaDBWindows, Linux and FreeBSD (both through Wine)Standalone2001
DbSchemaWise Coders GmbHSMBs and enterprisesProprietaryMS SQL ServerMySQLOracleFirebirdInterBaseSQL AnywhereNexusDBMariaDBSQLitePostgreSQLSybaseWindows, Linux, macOSStandalone2018
Enterprise ArchitectSparx SystemsSMBs and enterprisesProprietaryIBM Db2FirebirdInterBaseInformixIngresAccessMS SQL ServerMySQLSQLiteOraclePostgreSQLSybaseWindows, Linux (Wine), macOS (via CrossOver)Data modeling is supported as part of a complete modeling platform.2000
ER/StudioEmbarcadero (acquired by IDERA)SMBs and enterprisesProprietaryAccessSnowflakeMicrosoft Azure IBM Db2Informix, Hitachi HiRDB, FirebirdInterbaseMySQLMS SQL ServerNetezzaOraclePostgreSQLSybaseTeradataVisual Foxpro and others via ODBC/ANSI SQLWindowsStandalone1998
Erwin Data ModelerLogic Works (now Quest)SMBs and enterprisesProprietaryAccessIBM Db2InformixMySQLMS SQL ServerNetezzaOraclePostgreSQLSybase, and others via ODBC/ANSI SQLWindowsStandalone1998
MagicDrawNo MagicEnterprises, SMBs, personalProprietaryMS SQL ServerOracleMySQLPostgreSQLIBM Db2Windows, Linux, macOSStandalone1995
SQL Server Management StudioMicrosoftUn­knownProprietaryMS SQL ServerWindowsStandalone2005
ModelRightModelRightEnterprises, SMBs, personalProprietaryAccessMS SQL ServerOracleMySQLPostgreSQLIBM Db2WindowsStandalone2005
MySQL WorkbenchMySQL (An Oracle Company)SMBs - personalProprietary or GPLMySQLLinux, Windows, macOSStandalone2006
Navicat Data ModelerPremiumSoftSMBs and enterprisesProprietaryMySQLMS SQL ServerPostgreSQLOracleSQLiteWindows, macOS, LinuxStandalone2012
NORMA Object-Role ModelingTerry HalpinSMBs and enterprisesOpen source (CPL)MySQLMS SQL ServerPostgreSQLOracleIBM Db2WindowsVisual Studio Extension2005
Open ModelSphereGranditeEnterprises - SMBs - personalOpen source (GNU GPL3)MS SQL ServerMySQLPostgreSQLOracleIBM Db2Windows, macOS, LinuxStandalone with Data, UML, and process modeling2008
Oracle SQL Developer Data ModelerOracleEnterprisesProprietaryOracleMS SQL ServerIBM Db2Cross-platformStandalone2009
PowerDesignerSAPSMBs and enterprisesProprietaryAccessGreenplumApache HiveHP NeoviewIBM Db2InformixIngresInterbaseMySQLNetezzaNonStop SQLOraclePostgreSQL, Red Brick Warehouse, SAP business SuiteSAP HanaSAP Adaptive Server EnterpriseSAP IQSAP SQL AnywhereMS SQL ServerTeradataWindowsStandalone1989
Software Ideas ModelerDusan RodinaEnterprises, SMBs, personalProprietaryMS SQL ServerMySQLWindowsStandalone2009
SQLyogWebyog, Inc.Enterprises, SMBs, personalProprietaryMySQLMariaDBWindows and Linux (using Wine)Standalone2001
Toad Data ModelerQuest SoftwareSMBs and enterprisesProprietaryAccessIBM Db2InformixMySQLMariaDBPostgreSQLMS SQL ServerSQLiteOracleWindowsStandalone2005 (before this date known as CaseStu

Tuesday, November 19, 2024

inner join and outer join queries and results

 

 

--  // Create database //

 use db1;

show tables;

 

create table t1 ( id int , name varchar(44) ) ;

create table t2 ( id int , name varchar(44) ) ;

 

 

--  // insert data  into tables  //

 

insert into t1 values( 1, "x1");

 

insert into t1 values( 2, "x2");

insert into t1 values( 3, "x3");

insert into t1 values( 4, "x4");

 

select * from t1;

 

insert into t2 values( 1, "x1");

insert into t2 values( 2, "x2");

insert into t2 values( 5, "x5");

insert into t2 values( 6, "x6");

 

 

select * from t2;

 ------------------

 

-- // inner join //

select * from t1 join t2 on t1.id = t2.id

 

 

-- // left join //

select * from t1 left join t2 on t1.id = t2.id

 

 

-- // Right  join //

 

select * from t1 right join t2 on t1.id = t2.id

 

 

- // fuill outer  join //

 

 select * from t1 left join t2 on t1.id = t2.id

union

 select * from t1 right join t2 on t1.id = t2.id

 

 

150 ) Data model - Optimization

   Data model - Optimization   Diagnosing, Reporting, and Resolving a Data Model Problem Here is the complete, end-to-end process showing ho...