Add to Chrome
Add to Firefox
Add to EdgeUnderstanding the Cross-Domain Capabilities of Video-Based Few-Shot Action Recognition Models
Jun 03, 2024
Few-shot action recognition (FSAR) aims to learn a model capable of identifying novel actions in videos using only a few examples. In assuming the base dataset seen during meta-training and novel dataset used for evaluation can come from different domains, cross-domain few-shot learning alleviates data collection and annotation costs required by methods with greater supervision and conventional (single-domain) few-shot methods. While this form of learning has been extensively studied for image classification, studies in cross-domain FSAR (CD-FSAR) are limited to proposing a model, rather than first understanding the cross-domain capabilities of existing models. To this end, we systematically evaluate existing state-of-the-art single-domain, transfer-based, and cross-domain FSAR methods on new cross-domain tasks with increasing difficulty, measured based on the domain shift between the base and novel set. Our empirical meta-analysis reveals a correlation between domain difference and downstream few-shot performance, and uncovers several important insights into which model aspects are effective for CD-FSAR and which need further development. Namely, we find that as the domain difference increases, the simple transfer-learning approach outperforms other methods by over 12 percentage points, and under these more challenging cross-domain settings, the specialised cross-domain model achieves the lowest performance. We also witness state-of-the-art single-domain FSAR models which use temporal alignment achieving similar or worse performance than earlier methods which do not, suggesting existing temporal alignment techniques fail to generalise on unseen domains. To the best of our knowledge, we are the first to systematically study the CD-FSAR problem in-depth. We hope the insights and challenges revealed in our study inspires and informs future work in these directions.
Radar-based Materials Classification Using Deep Wavelet Scattering Transform: A Comparison of Centimeter vs. Millimeter Wave Units
Feb 08, 2022
Radar-based materials detection received significant attention in recent years for its potential inclusion in consumer and industrial applications like object recognition for grasping and manufacturing quality assurance and control. Several radar publications were developed for material classification under controlled settings with specific materials' properties and shapes. Recent literature has challenged the earlier findings on radars-based materials classification claiming that earlier solutions are not easily scaled to industrial applications due to a variety of real-world issues. Published experiments on the impact of these factors on the robustness of the extracted radar-based traditional features have already demonstrated that the application of deep neural networks can mitigate, to some extent, the impact to produce a viable solution. However, previous studies lacked an investigation of the usefulness of lower frequency radar units, specifically <10GHz, against the higher range units around and above 60GHz. This research considers two radar units with different frequency ranges: Walabot-3D (6.3-8 GHz) cm-wave and IMAGEVK-74 (62-69 GHz) mm-wave imaging units by Vayyar Imaging. A comparison is presented on the applicability of each unit for material classification. This work extends upon previous efforts, by applying deep wavelet scattering transform for the identification of different materials based on the reflected signals. In the wavelet scattering feature extractor, data is propagated through a series of wavelet transforms, nonlinearities, and averaging to produce low-variance representations of the reflected radar signals. This work is unique in comparison of the radar units and algorithms in material classification and includes real-time demonstrations that show strong performance by both units, with increased robustness offered by the cm-wave radar unit.
* 6 pages, 8 figures, accepted IEEE in Robotics and Automation Letters c. January 2022 associated video: https://www.youtube.com/watch?v=Mfohzvf7iuA
Modelling resource contention in multi-robot task allocation problems with uncertain timing
Feb 08, 2017
This paper proposes an analytical framework for modelling resource contention in multi-robot systems, where the travel times and task durations are uncertain. It uses several approximation methods to quickly and accurately calculate the probability distributions describing the times at which the tasks start and finish. Specific contributions include a method for calculating the probability of a set of independent normally distributed random events occurring in a given order, an upper bound on that probability, and a method for calculating the most likely and $n$-th most likely orders of occurrence for a set of independent normally distributed random events that have equal standard deviations. The complete framework is shown to be much faster than a Monte Carlo approach for the same accuracy in two multi-robot task allocation problems. This is a general framework that is agnostic to the optimisation method and objective function used, and is applicable to a wide range of robotics and non-robotics problems.
Weekly maintenance scheduling using exact and genetic methods
Oct 17, 2016
The weekly maintenance schedule specifies when maintenance activities should be performed on the equipment, taking into account the availability of workers and maintenance bays, and other operational constraints. The current approach to generating this schedule is labour intensive and requires coordination between the maintenance schedulers and operations staff to minimise its impact on the operation of the mine. This paper presents methods for automatically generating this schedule from the list of maintenance tasks to be performed, the availability roster of the maintenance staff, and time windows in which each piece of equipment is available for maintenance. Both Mixed-Integer Linear Programming (MILP) and genetic algorithms are evaluated, with the genetic algorithm shown to significantly outperform the MILP. Two fitness functions for the genetic algorithm are also examined, with a linear fitness function outperforming an inverse fitness function by up to 5% for the same calculation time. The genetic algorithm approach is computationally fast, allowing the schedule to be rapidly recalculated in response to unexpected delays and breakdowns.
Methods for Stochastic Collection and Replenishment (SCAR) optimisation for persistent autonomy
Jun 30, 2016
Consideration of resources such as fuel, battery charge, and storage space, is a crucial requirement for the successful persistent operation of autonomous systems. The Stochastic Collection and Replenishment (SCAR) scenario is motivated by mining and agricultural scenarios where a dedicated replenishment agent transports a resource between a centralised replenishment point to agents using the resource in the field. The agents in the field typically operate within fixed areas (for example, benches in mining applications, and fields or orchards in agricultural scenarios), and the motion of the replenishment agent may be restricted by a road network. Existing research has typically approached the problem of scheduling the actions of the dedicated replenishment agent from a short-term and deterministic angle. This paper introduces a method of incorporating uncertainty in the schedule optimisation through a novel prediction framework, and a branch and bound optimisation method which uses the prediction framework to minimise the downtime of the agents. The prediction framework makes use of several Gaussian approximations to quickly calculate the risk-weighted cost of a schedule. The anytime nature of the branch and bound method is exploited within an MPC-like framework to outperform existing optimisation methods while providing reasonable calculation times in large scenarios.
* Revision 1
Applying Gaussian distributed constraints to Gaussian distributed variables
Apr 20, 2016
This paper develops an analytical method of truncating inequality constrained Gaussian distributed variables where the constraints are themselves described by Gaussian distributions. Existing truncation methods either assume hard constraints, or use numerical methods to handle uncertain constraints. The proposed approach introduces moment-based Gaussian approximations of the truncated distribution. This method can be applied to numerous problems, with the motivating problem being Kalman filtering with uncertain constraints. In a simulation example, the developed method is shown to outperform unconstrained Kalman filtering by over 40% and hard-constrained Kalman filtering by over 17%.
Stochastic Collection and Replenishment (SCAR) Optimisation for Persistent Autonomy
Mar 07, 2016
Robots have a finite supply of resources such as fuel, battery charge, and storage space. The aim of the Stochastic Collection and Replenishment (SCAR) scenario is to use dedicated agents to refuel, recharge, or otherwise replenish robots in the field to facilitate persistent autonomy. This paper explores the optimisation of the SCAR scenario with a single replenishment agent, using several different objective functions. The problem is framed as a combinatorial optimisation problem, and A* is used to find the optimal schedule. Through a computational study, a ratio objective function is shown to have superior performance compared with a total weighted tardiness objective function, with a greater performance advantage present when using shorter schedule lengths. The importance of incorporating uncertainty in the objective function used in the optimisation process is also highlighted, in particular for scenarios in which the replenishment agent is under- or fully-utilised.
Stochastic Collection and Replenishment (SCAR): Objective Functions
Mar 07, 2016
This paper introduces two objective functions for computing the expected cost in the Stochastic Collection and Replenishment (SCAR) scenario. In the SCAR scenario, multiple user agents have a limited supply of a resource that they either use or collect, depending on the scenario. To enable persistent autonomy, dedicated replenishment agents travel to the user agents and replenish or collect their supply of the resource, thus allowing them to operate indefinitely in the field. Of the two objective functions, one uses a Monte Carlo method, while the other uses a significantly faster analytical method. Approximations to multiplication, division and inversion of Gaussian distributed variables are used to facilitate propagation of probability distributions in the analytical method when Gaussian distributed parameters are used. The analytical objective function is shown to have greater than 99% comparison accuracy when compared with the Monte Carlo objective function while achieving speed gains of several orders of magnitude.