In this paper, we will study modular Abelian varieties with odd congruence numbers by examining the cuspidal subgroup of $J_0(N)$. We will show that the conductor of such Abelian varieties must be of a special type. For example, if $N$ is the conductor of an absolutely simple modular Abelian variety with an odd congruence number, then $N$ has at most two prime divisors, and if $N$ is odd, then $N=p^��$ or $N=pq$ for some prime $p$ and $q$. In the second half of this paper, we will focus on modular elliptic curves with odd modular degree. Our results, combined with the work of Agashe, Ribet, and Stein, finds necessary condition for elliptic curves to have odd modular degree. In the process we prove Watkins's conjecture for elliptic curves with odd modular degree and a nontrivial rational torsion point.

22 pages, submitted to Math Annalen