Within the Internet service provider landscape, the residential gateway (RGW) plays a key role in service provision. The RGW should be an enabler for the provision of new and better services for residential users, but it is often instead an obstacle for innovation. This paper discusses how to improve the provision of innovative services and to increase the usability of residential networks by upgrading the residential gateway in alignment with the current paradigms of software-defined networking (SDN) and network function virtualization. In this approach, SDN contributes by providing a fine-grained control of the traffic and network function virtualization contributes by outsourcing traditional and specialized network functions running inside the RGW like routing or network address translation to the Internet service provider premises. Based on this approach, a management framework has been designed by considering 2 aspects: the involvement of the residential user in the management tasks through the provision of network management applications and the need to decouple network applications from the underlying SDN controller technology to encourage the development of innovative network applications. In addition, a virtualized management and networking domain has been defined to complement the approach and leverage cloud technologies. The advantages and challenges of this approach are analyzed based on a proof of concept development.