Perception Autonomy Engineer

Career Area:

Technology, Digital and Data

Job Description:

Your Work Shapes the World at Caterpillar Inc.

When you join Caterpillar, you're joining a global team who cares not just about the work we do – but also about each other.  We are the makers, problem solvers, and future world builders who are creating stronger, more sustainable communities. We don't just talk about progress and innovation here – we make it happen, with our customers, where we work and live. Together, we are building a better world, so we can all enjoy living in it.

Role Definition

Caterpillar’s Cat Technology team is seeking a Perception Developer to design, develop, and deploy perception software for autonomous and semi-autonomous machines used in construction. The role focuses on multi-sensor perception, computer vision, 3D point-cloud processing, sensor fusion, and production-quality software for safety-critical, resource-constrained platforms. The ideal candidate combines strong C++/Python development skills with hands-on experience building, optimizing, and validating perception algorithms from prototype through field deployment.

Responsibilities:

1. Design and implement perception algorithms for object and personnel detection, classification, segmentation, tracking, terrain understanding, localization, mapping, and obstacle assessment.

2. Develop camera, LiDAR, radar, GNSS, and IMU processing pipelines, including calibration, synchronization, sensor fusion, and uncertainty handling.

3. Translate research concepts and state-of-the-art models into robust, maintainable, production-quality C++ and Python software.

4. Optimize algorithms and neural-network inference for embedded CPU/GPU platforms using profiling, CUDA, TensorRT, and related acceleration technologies.

5. Build automated evaluation workflows, datasets, simulation scenarios, and test tools to measure accuracy, latency, robustness, and safety performance.

6. Debug and resolve perception issues using recorded data, simulation, hardware-in-the-loop systems, and field-machine testing.

7. Collaborate with systems, autonomy, controls, embedded software, validation, and safety teams to define interfaces and deliver integrated machine capabilities.

8. Participate in architecture reviews, code reviews, technical documentation, and continuous improvement of software engineering practices.

Desired Qualifications:

1. Bachelor’s or master’s degree in Computer Science, Computer Engineering, Robotics, Electrical Engineering, Mechatronics, or a related field.

2. Strong programming skills in modern C++ and Python, with experience developing and debugging software on Linux.

3. Practical experience with computer vision, deep learning, 3D geometry, point-cloud processing, or robotic perception.

4. Experience with one or more perception tasks such as detection, segmentation, classification, tracking, localization, mapping, or sensor fusion.

5. Working knowledge of PyTorch or TensorFlow and common perception libraries such as OpenCV, PCL, Eigen, or similar tools.

6. Understanding of camera, LiDAR, radar, GNSS, and IMU sensor characteristics, calibration, coordinate systems, and data quality considerations.

7. Experience with ROS/ROS 2, DDS, Git, unit testing, continuous integration, and software documentation practices.

8. Strong analytical, problem-solving, communication, and teamwork skills, with the ability to work across research and product-development teams.

Preferred Qualifications:

1. Experience deploying perception models on embedded or edge platforms such as NVIDIA Jetson, DRIVE, or comparable hardware.

2. Experience with CUDA, TensorRT, model quantization, profiling, and real-time performance optimization.

3. Knowledge of SLAM, visual or LiDAR odometry, Kalman filtering, probabilistic estimation, and graph-based spatial data structures.

4. Experience developing perception systems for autonomous vehicles, ADAS, mobile robots, mining, construction, or off-highway equipment.

5. Experience with simulation, synthetic data, dataset curation, annotation workflows, MLOps, or field validation.

6. Publications, patents, or demonstrated implementation of state-of-the-art perception methods are an advantage.

Be Part of What’s Next in Autonomous Construction Sites

Construction autonomy is one of the most complex challenges in applied AI, and at Caterpillar, advancements in physical AI, simulation, sensing, and edge computing are turning things that once felt impossible—intelligent machines operating in dynamic jobsites—into reality.

Our connected ecosystem brings together massive volumes [BH1.1][LR1.2]of high-quality data to create a foundation where engineers like you can build and deploy against.

If this work motivates you, we invite you to join our team. In these roles, you’ll work at the intersection of the physical and digital worlds. You’ll help design and deliver intelligent systems that enable machines to perceive their environment, make informed decisions, and support safer, more productive operations.

Apply today to build the new era of construction autonomy at Caterpillar.

 

Posting Dates:

October 13, 2026 - October 26, 2026

Caterpillar is an Equal Opportunity Employer.  Qualified applicants of any age are encouraged to apply

Not ready to apply? Join our Talent Community.