Transforming industry

Shipping/ports

The Port of Brisbane and the Port of Botany Bay in Sydney both function with a minimum of human input.

All the imports that come through these ports—much of our furniture, electronics, computers and appliances—are delivered from ship to shore by cranes and then shuffled around the terminal by automated straddle carriers (big four-legged frame structures with wheels that can carry a shipping container) and trucks.

The Port of Brisbane kicked things off by putting 35 automated straddle carriers (AutoStrads) into action in 2007, after a 10-year development period. In 2015, 50 AutoStrads took over operations at Sydney’s Port Botany, making it the largest outdoor autonomous system in the world. Just one person is required to oversee the workings of the entire container yard.

So, how do these robots do their work? There were three key components involved in building the AutoStrads:

• development of a navigation system that was reliable and guaranteed to be accurate
• control of a large, heavy vehicle that was capable of carrying heavy loads 
• design of an entire autonomous system that is predictable, efficient, repeatable and predictable.

The AutoStrad vehicle was modelled on a standard manned straddle carrier. It’s around 10 metres high, 3.5 metres wide and 9 metres long, and weighs around 65 tons. It’s capable of carrying loads of up to 50 tons, and runs on a diesel–electric engine.

The AutoStrad robot’s ‘brain’ consists of a navigation system and a pilot controller. These feed information to what’s called the task controller, which then provides commands to the machine controller. The machine controller is responsible for the actual functions and actions of the AutoStrad. A separate safety system incorporates collision detectors and monitors the vehicle’s ‘health’.

The navigation system is twofold, with two independent systems that operate in fundamentally different ways. Having two systems that do not share any common information ensures the integrity of the system—any mistakes one system makes will be identified and compensated for by the other.

The first system uses a millimetre wave radar sensor that can operate through rain and airborne particulate matter. It communicates with beacons attached to the lighting towers of the terminal, receiving between four and 10 bearing observations per second. The second part of the navigation system is a real-time-kinematic global positioning sensor.

Each of the sensor systems are coupled with two different rate sensors that detect the AutoStrad’s speed, which is also important for determining the vehicle’s location and tracking its movement.

Safety is paramount in the construction of an automated system. The AutoStrads only operate if they receive a ‘heartbeat’ radio signal; without it, they shut down. The automated area of the port is strictly fenced, and any unauthorised entry will shut off the heartbeat signal. Individual AutoStrads have a continuous monitoring system keeping track of all the vehicle’s operations, and a laser on each corner to detect any obstacles and avoid collisions.

The end result is a level of automation that not only improves the safety of the port terminals—with no humans moving around the terminal, no one’s there to get injured—but also the efficiency and productivity of the business.

Mining

But what if we’re dealing with a system that’s a little more complicated than moving a bunch of hard metal blocks around a clearly defined space?

Mining operations occur on a huge scale—there are huge trucks, huge drills and huge amounts of rock that need to be blown up and collected. We’re no longer talking about square boxes and straight lines. This is an environment where automation can not only provide a means of performing jobs in a safer and more efficient way, but can do things that couldn’t be done before.

In Western Australia, there are now 14 mines, all run by Rio Tinto, and all operated by remote from a control centre near Perth Airport.

These mines have automated driverless trucks, equipped with road-edge guidance systems that measure the distance from their windows to the side of the road, keeping them on track. They use the Simultaneous Localisation and Mapping (SLAM) system to navigate underground, and have radar guidance systems on the front bumper bars that detect any obstacles in their path. 

• Robot navigation: Simultaneous Localisation and Mapping

  One of the crucial challenges for robotics is to enable robots to navigate their way successfully through an unknown environment. One method developed to address this is known as the Simultaneous Localisation and Mapping (SLAM) problem. Researchers have been developing algorithms that can be used by a robot to describe and interpret its environment. 

  These algorithms are essentially based on using correlations between landmarks identified by the robot’s sensors and the robot’s own location to put together a probabilistic map of its environment and its location within it.

  The robot measures various landmarks as it travels through the environment, and records the locations of these landmarks relative to itself and, even more importantly, to each other. For example, the robot, while in location 1, takes a measurement of landmark A and landmark B. It then goes on to location 2, and takes another measurement of landmark B (it uses a re-association process to make sure that this is landmark B again, and not an entirely new landmark) and a measurement of landmark C. It now has information about the location of landmark C relative to landmark B. It already has information about the location of landmark B relative to landmark A, so it’s able to use this to figure out how the locations of landmarks C and A also relate to each other.

  Each new measurement provides information that can be related back through the series of previous measurements, updating and strengthening the existing data. The robot also knows the trajectory and distance that it travelled to get from location 1 to location 2, and so this data, too, feeds into the map of relative locations.

  As more and more measurements are made, the overall picture of the environment becomes more detailed and accurate. Any errors in the measurements arise only from imperfect knowledge of where the robot itself is located, while knowledge of the relative locations of the landmarks is highly accurate. This means that, while the robot might have difficulty pinpointing one particular landmark on its own, the overall map it builds is very comprehensive. 

Even more important than the automated trucks on a mine site are the automated drills. Drilling through bodies of rock is an essential part of mine operations, and it's used for both exploration, to see what's in the rocks, and also in the next stage—blasting the rocks to extract their contents. Drilling through rock is very expensive, but with automated drill systems can deliver even more information than just the physical chunks of rock they pull up. Equipped with sensors and other equipment, they can perform measurements of the rocks’ (and the ore bodies embedded within) chemical and physical characteristics that can provide reams of useful data that wasn’t obtainable during the exploration stage.

So while the automated equipment itself provides a safer and more efficient way to conduct the mine’s operations, the real payoff comes from the way it enables the manipulation of information to make a single integrated picture of the site’s overall geology and the ore body.

Transforming agriculture

Agriculture is another operation in which effective use of information can be the key to productive practices. Automation provides the capacity to gather this information efficiently and on a scale that isn't otherwise possible. Data about soil type and quality, rainfall, crop yields and vegetation type can all be collected and compiled to contribute to more effective farming practices.

Monitoring the farm

While satellite imagery can provide a resolution of around 50 centimetres per pixel GLOSSARY pixelA tiny dot of colour that when combined with numerous others form a graphic image , an unmanned aerial vehicle (UAV) survey can provide a much higher resolution of 1–10 centimetres per pixel. It can also fly on demand whenever the information is required.

UAVs (often called drones) can carry a range of different sensors and equipment. Digital SLR cameras, thermal sensors and 6-band multispectral sensors (these create images of the vegetation using wavelengths of the electromagnetic spectrum other than visible light) can be used to map vegetation types. Subtle differences in the way various types of plants reflect light, or appear when imaged using near-infrared (or other wavelengths on the electromagnetic spectrum) sensors, can be used to interpret changes in vegetation, such as distinguishing between crops and weeds like prickly acacia or alligator weed.

Farmers can also monitor plant health. Multispectral sensors image the plants beyond the visible range. This can identify stress in a plant before its effects are visible to the human eye, by which time it’s often too late for the plant. The more chlorophyll in a leaf, the more near-infrared light it will absorb. As leaves dry out, and lose their chlorophyll, a near-infrared image is very different to that of a healthy leaf.

Doing things on the farm

Other robots, called Ladybirds, have been developed. These can travel methodically through a crop field and, using a camera mounted underneath its base, find and identify weeds. Having found them, it precisely sprays those weeds—and only the weeds!—reducing the need for widespread use of herbicides. 

Another such helper robot, the Agbot (II and III) was created by researchers at the Queensland University of Technology. This is a three-wheeled rover-type robot, that can carry out weed management, fertiliser and herbicide application and seeding functions. It's equipped with multiple, inexpensive sensors and uses complex algorithms to combine the different types of data the sensors collect. Cameras feeding images to a computer vision system are used to navigate and for obstacle avoidance, but also for weed detection and to control the rate of herbicide application.

Transforming retail

It all started back in 2006 in a warehouse of the stationery store Staples, when they began using 60 robots designed by a company called Kiva Systems to help shuffle stock around the warehouse. By 2007, they were using 500 robots. In 2012, Kiva Systems was bought by the online retail store Amazon.

The Kiva bots perform the job of finding items on the warehouse shelves, then bringing those (portable) shelves to a human operator so they can pack the items to be shipped. This task of item collection used to be done (and, in many warehouses, still is) by a person physically walking around the warehouse to the items’ locations. Some of Amazon’s warehouses (when you include extra mezzanine areas) cover an area of slightly more than 22 hectares— equivalent to around 20 cricket grounds.

In a warehouse equipped with Kiva bots, it’s the bots that do all the ‘walking’. They scoot along the warehouse floor, pick up portable shelves (pods) containing the desired item and carry the pod over to a central ‘picking’ area staffed by humans. The human just needs to select items from the pods that the robot has delivered to them.

The Kiva bots are little square contraptions with two wheels, which enable them to either travel in a straight line or spin around on the spot. They have infrared detectors for avoiding obstacles. Cameras underneath them read QR codes on the ground that tell them where they are in the warehouse. A central computer system controls them all using WiFi.

Amazon’s warehouses now have around 30,000 Kiva robots working in 13 different locations across the United States.

Again, information is the key. The storage system in the warehouse is now a dynamic one, with no fixed shelf locations. Obviously, operators and managers need to have a record of what items are stored in which pods, and where exactly those pods are located within the warehouse. The Kiva bots need to manage their delivery to the picking area in a time-effective manner as, for orders with more than one item, chances are slim that all items will be stored on the same shelf.

• The Kiva bot process

  The process in a warehouse with a picking area goes something like this:

  • Order submitted online.
  • Order received; pod containing item is identified.
  • Kiva bot sent to fetch pod. The bots can only drive north, south, east or west, and are programmed to ‘tunnel’ underneath stationary pods as much as possible, leaving driving areas clear. 
  • The bot finds the pod.
  • The bot lifts the pod off the floor using a corkscrew motion. The pod is lifted just high enough so that the bot can carry it.
  • The bot travels through the local ‘streets’ to the ‘highway’, a broader area of more space leading to the picking area.
  • The bot queues at the correct picking station. This must be carefully timed to avoid having too many bots queuing at once.
  • The picker takes the item.
  • The bot returns the pod to storage. The pod’s resting place within the warehouse is determined by what’s left on the pod—if there’s not much left on it, the bot will take it further towards the back of the warehouse.
  • The bot un-corkscrews to drop the pod back to the floor.
  • The bot recharges.

Information is key

Crucial to all of these automated systems is the control of information. Compilation of multiple, and often enormous, datasets and (often) the processing that data in real time is what makes the robots so effective at what they do. You’ll be able to read more about the power of data, and what sophisticated computing systems can do with it, in another Nova topic down the track …

We’re still some way from the robot companions or combatants of our sci-fi stories. But that’s not to say that robots aren’t already making their presence felt. Their capabilities are indispensable to the way many industries conduct business.