Azure Data Engineer Interview Questions

 azure databricks and ADf , Synapse questions

1. What is the difference between narrow and wide transformation

2. Explain the archtitecture of pyspark

3. Can we execute notebook activity in adf, can we pass parameters to it

4 Do you know autoloader, and structured streaming

5. What is delta tables, how do they work 

6. How do we create delta tables in serverless pool and powerbi users can access it

7. what is integration runtime, where can we install it

8. Can we use SHIR inside mapping data flow

9 How do you connect to on premise source from ADF

10. How to create a pipeline that fetches data from two databases from two different schema , two tables

11. How can you load a huge table from SQL server in ADF activity

12. How to do SCD type 2 using ADF

13. Difference between type 2 and type 3 SCD

14 Read a csv file, Create a delta table using pyspark

15. Join two datasets in Pypark

16. SQL Query question on Window functions

17, Read a csv file, skip 5 rows of it, create schema manually

18 how do you handle bad records in spark, what are the different read modes

19 How do you dynamic mapping on the go in copy activity if source and target column names are different

20 Code to append two list in python, resulting string shouldn't contain duplicate 

21 GetMetadata activity what are its properties

22 Lookup activtity - How will you use its output

23 Write python code to Find a number is disynys or not : 135 is disysns because 135=1**1 + 3 **2 + 5 **3

24 Real time example of star and snowflake schema

25 Write code to perform autoloader operation in PYSPARK

26 How to load a 2gb file using copy activity 

27 Can you call one notebook from other notebook , if yes how and how will you pass parameters to it 

hint - using dbutilis.text.widget

28 How to receive parameters from ADF execute Notebook activity

29 How do you manage the cost of the cluster if it has to run autloader activity to check file arrival 

30 SSIS questions Blocking and Non Blocking transformation which is better

31 What is distribution in Synapse

32 Delta table working - how do they function

33 Activities used in your ADF project for eg Web activity