Elastic databases have grown in popularity over conventional databases in recent years due to their ability to be allocated with sufficient capacity for peak load. Especially with the support of the cloud platform, which provides flexible resources and low cost, elastic databases on the cloud show their excellent potential in scalability, flexibility, and accessibility. However, the interaction between the cloud layers of virtual machines (VMs) and databases further complicates the issue of cloud configuration to adapt to dynamic workloads. In this paper, I explore a framework for a self-configured elastic database that can optimize the cloud configuration and adaptively allocate resources under the constraints of databases' Service Level Agreement (SLA). At the core of the framework is a Deep Q learning approach, which combines the advantages of Reinforcement Learning (RL) and Deep Learning (DL). The framework is built on Amazon Web Service (AWS)'s cloud environment and uses MySQL database for its high availability replication mechanism. Experimental results on the TPC-W benchmark demonstrate that with the implementation of Deep Q learning, the elastic database reduces SLA violation by more than 90\%, in the response to the steep slope of workload change.