Hardware-aware Transformable Architecture Search with Efficient Search Space

While Neural Architecture Search (NAS) discovers the optimal topology structure, Transformable Architecture Search (TAS) aims to search for the best width and depth, which is more challenging due to the larger search space. Since FLOPs is inconsistent with the actual latency, hardware-aware TAS uses the inference latency to evaluate the efficiency. However, most existing work focuses on the search strategy, ignoring the critical role of the search space in affecting the actual efficiency. Motivated by it, we study hardware-aware TAS by considering the search space, to the best of our knowledge, for the first time. We propose a hardware-aware transformable architecture search (HTAS) framework to discover the optimal architecture for different hardware. The core of our method is a novel hardware-aware search space, which provides efficient channel choices for the search strategy to sample efficient architectures. Experiments on CIFAR datasets demonstrate the superiority of HTAS over the state-of-the-art method.