Find Jobs
Hire Freelancers

Software Engineer I

$35-50 USD / hour

Închis
Data postării: circa 4 ani în urmă

$35-50 USD / hour

Develop and validate solution architecture to support business requirements Develop Spark applications to read transaction data and process business rules to report errors and transaction summary Load and transform large sets of structured and semi structured. Read data from Kafka and process data using Spark. Adopt innovative architectural approaches to leverage in-house data integration capabilities. Analyze existing processes and prepare functional and requirements documents. Develop multiple spark streaming and core jobs with Kafka as a data pipe-line system. Hive external tables on HBase using Insert Overwrite with S3 as data storage. Troubleshoot developed Spark jobs. Manage and review Yarn Application Logs, Spark Event Logs and Metrics sink CSV files. Improve the performance and optimization of the existing jobs in Spark. Develop solutions by utilizing commercial and open source software including Minifi and Nifi to interface big data and relational solutions.
ID-ul proiectului: 23958059

Despre proiect

16 propuneri
Proiect la distanță
Activ: 4 ani în urmă

Vrei să câștigi bani?

Avantajele de a licita pe platforma Freelancer

Stabilește bugetul și intervalul temporal
Îți primești plata pentru serviciile prestate
Evidențiază-ți propunerea
Te înregistrezi și licitezi gratuit pentru proiecte
16 freelanceri plasează o ofertă medie de $45 USD/oră pentru proiect
Avatarul utilizatorului
Hello. I have a masters degree in AI and have worked with Hadoop, Spark, YARN, ... on a lot of projects. I can share some demos if you are interested. Thanks, Helmot
$35 USD în 40 zile
4,8 (221 recenzii)
8,2
8,2
Avatarul utilizatorului
Hi there, Being an expert in Python and matlab programming I can do this project for you. please leave a message on my chat so we can discuss the budget and deadline of the project. Also, have a look at my profile for my reviews and portfolios. Thanks Note: This is not an automated bid and i'm ready to discuss your project right now. .. .
$50 USD în 98 zile
5,0 (22 recenzii)
6,3
6,3
Avatarul utilizatorului
Hi, there. I am a senior Python & Web developer and my main skills are Django and Flask. Fully experienced in each step of web development - design, development, deployment and maintenance, and mastered management of various kinds of DBs including MySQL, MongoDB, PostgreSQL, etc. Especially, I have experience in Big Data Processing using Spark and Hadoop. If you're interested, please have a discussion with me. Thanks.
$40 USD în 40 zile
4,8 (5 recenzii)
4,9
4,9
Avatarul utilizatorului
Hi, I have 9 years of experience and working on hadoop, spark, Kafka, nosql, java, BI tools(tableau, powerbi), cloud(Amazon, Google, Microsoft Azure)... Done end to end data warehouse management projects on aws cloud with hadoop, hive, spark and presodb. Worked on multiple etl project like springboot, angular, node, PHP, Kafka, nifi, flume, mapreduce, spark with XML/JSON., Cassandra, mongodb, hbase, redis, oracle, sap hana, ASE.... Many more. Let's discuss the required things in detail. I am committed to work done and strong in issue resolving as well. Thanks
$50 USD în 30 zile
5,0 (7 recenzii)
4,0
4,0
Avatarul utilizatorului
Hello! I'd like to optimize information aggregation. I'm familiar with python and theory of probabilities. I'll do the job blazingly fast. Please, give me a try!
$56 USD în 80 zile
5,0 (3 recenzii)
4,0
4,0
Avatarul utilizatorului
Highly effective Data Engineer and Data Scientist with over 25 years of experience. I am specilaizing Big Data and have extensive Python / Spark experience both in PySpark and Scala also in MLLib. I developed a number of ML models including linear regression, logistic regression and multi classfication using various libraries. TECHNICAL SKILLS • Big Data (Hortonworks and Cloudera) – Spark(PySpark, Scala), Kafka, Hive,Impala, NiFi, HDFS, Sqoop, Ranger, Yarn, Solr, SAM, Schema Registry, SuperSet, Druid • Language: Python, Scala, R, JavaScript • Data Visualization – Tableau, PowerBI, OBIEE, DOMO • Plunk & ELK Stack – ElasticSearch, Logstash, Filebeat, Kibana • AWS – S3, RefShift, DynamoDB, Athena, Kinesis, EMR, Aurora, Glue • Azure – Data Warehouse, Polybase, SQL Server, HDInsight, SSIS, SSAS • ETL – Informatica Power Centre, Informatica Big Data Management (BDM), SSIS • Data Science & Engineering – R / RStudio / SparkR Packages: dplyr, ggplot2, stringr, plyr, carrat, SparkR, NLP, tibble, TensorFlow, curl, Python / PySpark, MLLib • Libraries: MLLib, NumPy, SciPy, Pandas, Matplotlib, Seaborn, SciKit-Learn • DBMS / OLAP – Oracle, SQL Server, TeraData, MySQL, Postgres, Essbase, SSAS • Data Modelling – Kimball, Vault • Machine Learning / Statistical Modelling - Linear Regression, ?Logistic Regression, Classification and Regression Trees, Naive Bayes, ?K-Nearest Neighbors
$55 USD în 40 zile
5,0 (2 recenzii)
4,0
4,0
Avatarul utilizatorului
I am currently working on each item which you described. I delivered end to end solution including kafka, spark, hive, nifi and all other required hadoop eco system components. Please let connect or have a call to discuss more in your requirements and also if you like with skills then only we move forward. Incase if you need any demo/POC work, I am ready to take that as well. I hope we will be connecting soon, I am available on my mobile, so feel free to connect at any time.
$44 USD în 20 zile
5,0 (2 recenzii)
2,2
2,2
Avatarul utilizatorului
Hi there, I have quite some experience in most of the big data tech out there (Hadoop, Spark, Kafka, Cassandra, ...), since I have been specializing on Data Management, Data Consolidation etc. for quite a long time, now. All in all, I am a true Full-Stack developer (leveraging mainly Node.JS, Python, Java, Ruby, etc.) and can thus maintain the full stack, build a GraphQL layer on top of the data, in order to allow easy access to it or build some frontend for it, as well. Additionally, I have quite some experience on the Business side of things, so I know how to manage a project (even if it is only myself working on it), communicate with key stakeholders and preparing documentation for technical, but also non-technical users. I would love to have a talk with you, since I am always searching for long-lasting partnerships, so it's got to be right, of course! P.S.: This seems to be a REALLY exciting project, so I just want to make sure to point out, that I think that I would be perfect to tackle this, even though they probably all say that, but I would really love a shot at this!
$50 USD în 40 zile
0,0 (0 recenzii)
0,0
0,0
Avatarul utilizatorului
I have developed country wide log parsing application using Apache Spark on Hadoop. I have experienced memory issues deeply. Sqoop Metastore Oozie workflow pipelines were used. Machine Learning is another side of thmy Big Data jobs. I use spring boot and Vue js for presentation layer. As a Software engineer I am working in a CMMI level 4 organisation.
$43 USD în 40 zile
0,0 (0 recenzii)
0,0
0,0
Avatarul utilizatorului
Have the experience of over 5 years in developing Big data pipelines over multiple technologies, hold certification in Hortonworks and Cloudera platforms. Currently employed full time as analytics application engineer with large corporation. Can contribute 20 hours per week
$40 USD în 20 zile
0,0 (0 recenzii)
0,0
0,0
Avatarul utilizatorului
Hi. I have a strong background in big data and Hadoop. I have worked as a Senior Data Platform Engineer and on technologies like spark, AWS, mapreduce, hbase, hive, Google bigquery, Kafka, Google cloud platform. I would love to discuss more about the position.
$55 USD în 30 zile
0,0 (0 recenzii)
0,0
0,0

Despre client

Steagul UNITED STATES
Los Angeles, United States
0,0
0
Membru din feb. 17, 2020

Verificarea clientului

Alte proiecte ale acestui client

Integration Engineer
min $50 USD / hour
Mulțumim! Ți-am trimis prin e-mail linkul pe care trebuie să-l accesezi pentru a revendica creditul gratuit.
A apărut o eroare la trimiterea e-mailului. Încearcă din nou.
Utilizatori înregistrați Totalul proiectelor postate
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Se încarcă previzualizarea
S-a oferit permisiunea de depistare a locației.
Ți-a expirat sesiunea pentru conectare sau te-ai deconectat. Conectează-te din nou.