This paper contributes to the ongoing debate on the plausibility of tacit collusion between sellers in algorithmic marketplaces, which can be detrimental to customers and social welfare. We study a broad class of assortment decisions routinely made by sellers on online platforms, including which products are offered to customers, at what price, and how are they displayed. In this context, algorithmic decision-support tools are extensively studied in the operations literature and widely adopted in practice. We propose simple notions of collusive outcomes to describe an "optimal form" of collusion between sellers under full information. While computing such collusive outcomes is NP-hard, we develop a polynomial-time approximation scheme, showcasing the computational tractability afforded by our solution concept. Our main contribution is to establish that collusive outcomes can be tacitly and near-optimally reached under very limited prior market information. Surprisingly, we show that a simple variant of epsilon-greedy -- a commonly used class of learn-and-earn algorithms -- is able to dynamically learn a collusive outcome without any form of explicit communication that is prohibited by antitrust laws. This algorithm asymptotically attains a collusive outcome with a worst-case expected regret of O(T^{2/3} log T) over T periods against the full-information benchmark. These findings give theoretical support to the concerns expressed by academics and policymakers in other fields.