Add Redshift Data Source In AWS Glue Catalog

Поделиться
HTML-код
  • Опубликовано: 2 окт 2024

Комментарии • 17

  • @Rohitshrimal
    @Rohitshrimal Год назад

    Hi thanks for the video, I am new to AWS, is there a way to access this table via athena ? what are the use cases where we might feel the need to add redshift table to glue catalog? Thanks in advance

  • @gauranshijohari4468
    @gauranshijohari4468 Год назад +1

    Your video was a saviour!!

  • @MuhammadSaad-kb7op
    @MuhammadSaad-kb7op Год назад

    I set up an IAM role called "AWSGlueServiceRole."
    Sir, I tried running the crawler on AWS Glue, but an error showed up, "Crawler cannot be started. Verify the permissions in the policies attached to the IAM role defined in the crawler."
    Can you please help me resolve this issue? Alternatively, Sir, could you create a video on how to add an RDS MySQL data source in the AWS Glue Catalog?

    • @DataEngUncomplicated
      @DataEngUncomplicated  Год назад

      Hi Muhammad, what permissions did you add? Sounds like it might be a permission error

  • @nehalverma1444
    @nehalverma1444 Год назад

    When I test my glue connection it always fails. I have created s3 Endpoints, even security group inbound ruled are all traffic allow from anywhere, my role has glueservice role permission. Everthing seams fine but why it fails. Please help

  • @MuhammadSaad-kb7op
    @MuhammadSaad-kb7op Год назад

    Sir, can you create a video on AWS Glue 'Adding RDS MySQL Data Source to the AWS Glue Catalog'?

    • @DataEngUncomplicated
      @DataEngUncomplicated  Год назад +1

      Sure I'll add this idea to my video list thanks for the suggestion!

  • @johnychandrach
    @johnychandrach Год назад

    Thank you for the video, it helped me understand how we can connect Redshift tables with Glue.
    The crawler runs fine, but when using the catalog table created by the crawler in the Glue job I'm getting the following error.
    SdkClientException occurred: com.amazonaws.SdkClientException: Unable to execute HTTP request: Connect to aws-glue-assets-.......... failed: connect timed out
    Any inputs?

    • @DataEngUncomplicated
      @DataEngUncomplicated  Год назад

      No problem, Hmm it sounds like it could be an vpc issue. I would check to make sure your glue job has access to the redshift vpc

  • @waynelo8088
    @waynelo8088 7 месяцев назад

    Hi, I cannot query the cataloged redshift tables through Athena.
    Can we somehow make the cataloged tables queryable through Athena?
    If not what's the use case for adding the redshift tables into Glue cataloge. i.e. What's the purpose of the result of this video?

    • @DataEngUncomplicated
      @DataEngUncomplicated  7 месяцев назад

      Hey, if you have redshift you are already paying for compute so curious why you want to go through Athena vs redshift to do this? You can look into this option docs.aws.amazon.com/athena/latest/ug/connectors-redshift.html
      The purpose of having a redshift table in a glue catalog so you can access your redshift tables in glue jobs or lambda functions using the aws sdk for pandas library.

    • @waynelo8088
      @waynelo8088 7 месяцев назад

      I was exploring using Athena as a central interface to provide access to all our data assets via the Glue catalog.
      I don't want to move the data out of Redshift again, just make it queryable from Athena as needed.
      @@DataEngUncomplicated

  • @lukmansetiadi
    @lukmansetiadi Год назад

    the crawler is successfull, but somehow can not query against those table

  • @rajatpathak4499
    @rajatpathak4499 Год назад

    great