Ben Affleck and Jennifer Lopez enjoyed ‘pretty quiet holiday’ on Christmas as they spent time with family and each other