Comments for Big Data Processing

Comment on Airflow Operational Dashboard using Bigquery and Looker Studio by priya

priya — Tue, 21 Jan 2025 21:10:03 +0000

Hi , how can I modify the query to get the how long a task run ?

Comment on GCP – Create Custom Bigquery Linage using DataCatalog Python API by Visarn Patton

Visarn Patton — Wed, 02 Oct 2024 11:06:36 +0000

Why am I getting a credentail error when I tried your code

Comment on GCP – Create Custom Bigquery Linage using DataCatalog Python API by Visarn Patton

Visarn Patton — Wed, 02 Oct 2024 11:05:16 +0000

Why is it I am getting credentials errors when I tried your code?

Comment on GCP – Bring CDC data using Debezium by Sam

Sam — Sun, 05 Feb 2023 22:18:19 +0000

Thank you sharing the info. I wonder if you have tried to use Microsoft SQL Server as the source? Or how can I setup the application.properties file for it?

Comment on What is vectorization in hive? by Mouli Shareef Shaik

Mouli Shareef Shaik — Wed, 11 Aug 2021 02:09:37 +0000

Very good explanation.

Comment on What is Predicate PushDown in Hive? by shyam kumar

shyam kumar — Wed, 24 Mar 2021 04:45:53 +0000

Why Duration is high when applied PPD?

Comment on What is vectorization in hive? by CD

CD — Thu, 24 Sep 2020 02:54:34 +0000

excellent explanation

Comment on Update Hive Table by Ashif Ahamed

Ashif Ahamed — Fri, 04 Sep 2020 22:00:17 +0000

Pls modify the overwrite queries :

Equality operator is used instead of inequality operator in the first insert overwrite query

Comment on What is Predicate PushDown in Hive? by Harshul

Harshul — Tue, 04 Aug 2020 07:43:42 +0000

Hi,Can you please tell me how can i view the same DAG in my hive?

Comment on Hive Lateral view explode vs posexplode by Mohanpraveen

Mohanpraveen — Tue, 12 May 2020 17:22:19 +0000

Hi ,

For example if I have in first AAA row only one string in a list both phone number and city but in second row you have two phone numbers and two cities like mix of both multiple and single how can we use pos explode in that case?