Polynomials are common algebraic structures, which are often used to approximate functions, such as probability distributions. This paper proposes to directly define polynomial distributions in order to describe stochastic properties of systems rather than to assume polynomials for only approximating known or empirically estimated distributions. Polynomial distributions offer great modeling flexibility and mathematical tractability. However, unlike canonical distributions, polynomial functions may have non-negative values in the intervals of support for some parameter values; their parameter numbers are usually much larger than for canonical distributions, and the interval of support must be finite. Hence, polynomial distributions are defined here assuming three forms of a polynomial function. Transformations and approximations of distributions and histograms by polynomial distributions are also considered. The key properties of the polynomial distributions are derived in closed form. A piecewise polynomial distribution construction is devised to ensure that it is non-negative over the support interval. A goodness-of-fit measure is proposed to determine the best order of the approximating polynomial. Numerical examples include the estimation of parameters of the polynomial distributions and generating polynomially distributed samples.