This paper introduces the Distributed NFV Orchestrator (DiNO). DiNO integrates and orchestrates Software-Defined Networking (SDN) and Network Function Virtualization (NFV) in a physically and logically distributed manner and optimizes their service quality and resource utilization. In the DiNO architecture, Virtualized Network Functions (VNFs), including servers, middleboxes, and QoS, can be dynamically deployed and discharged by carefully monitoring the service state, estimating their demand, and examining the satisfaction of Service Level Agreements (SLAs). We introduce the serving IP address allowing multiple VNFs of the same service to be served with a consistent IP address and loadbalancing. The serving IP address also enables VNFs to serve the request on the basis of any parameter such as hop-count, response time, and bandwidth utilization. The placement of VNFs is optimized on the basis of the choice of algorithm, where we use a Greedy algorithm for the preliminary study. DiNO has been prototyped using Open Stack, Floodlight, and Open vSwitch. The contribution of DiNO is 1) to introduce high flexibility into NVF deployment, 2) to aggressively reduce the deployment cost, and 3) to maintain the service quality by serving end users off-the-core.