In this letter, we study the problem of decentralized motion planning of robot swarms under high-level temporal logic specifications with a top-down approach. We use Swarm Signal Temporal Logic (SwarmSTL) to express swarm-level specifications. By encoding SwarmSTL formulas as mixed binary-integer constraints on the swarm features, the motion planning problem is formulated as a mixed-integer quadratic programming (MIQP) problem. We develop a decentralized Branch and Bound (B&B) algorithm with a node decentralization scheme such that the nodes in the B&B tree can be processed in parallel with communication among the agents and the agents can achieve consensus on the solution. Also, several search strategies to accelerate the decentralized B&B algorithm are proposed, and the performance improvements are presented. We evaluate the proposed algorithm using a supply transportation example with different formulas.