Model Monitoring and Review Team Lead - Analytics Manager 4 in San Francisco, California | DiversityInc Careers

Model Monitoring and Review Team Lead - Analytics Manager 4

Job Description

At Wells Fargo, we want to satisfy our customers’ financial needs and help them succeed financially. We’re looking for talented people who will put our customers at the center of everything we do. Join our diverse and inclusive team where you’ll feel valued and inspired to contribute your unique skills and experience.

Help us build a better Wells Fargo. It all begins with outstanding talent. It all begins with you.

Data Management and Insights (DMI) is transforming the way that Wells Fargo uses and manages data. Our work enables Wells Fargo to empower and inform our team members, deliver exceptional experiences for our customers, and meet the elevated expectations of our regulators. The team is responsible for designing the future data environment, defining data governance and oversight, and partnering with technology to operate the data infrastructure for the company. This team also provides next generation analytic insights to drive business strategies and help meet our commitment to satisfy our customers’ financial needs.

The NLP Data Science team in Enterprise Analytics and Data Science (EADS) is responsible for developing and deploying NLP, machine learning, and AI solutions for key strategic Enterprise initiatives. The team is looking for an experienced analytics professional to manage the Model Monitoring and Review Team. As the manager for the team, you will be responsible for building and managing a team of data scientists to design, develop, and implement the AI-NLP model monitoring framework and monitoring activities including but not limited to model performance tracking, KPI tracking and reports, thresholds and deviation tracking in ways to ensure the procedures and outputs meet corporate model risk policy and requirements and lines of business needs. In addition, you will be responsible for working with corporate model validation team to establish a working model that consists of standardized processes, repeatable modules, and approved artifacts to scale up the model review and model validation process. You and your team will be working with enterprise data scientists, corporate risk model validation teams, and the business customers to improve efficiency in the model review processes.


• Manage and grow a team of data scientists responsible for AI-NLP model review and monitoring
• Partner with corporate risk teams to understand model risk policy and its implications to model review and monitoring
• Establish and implement a standardized model monitoring process that adheres to policy standard
• Develop and implement a standardized internal model review/peer review process
• Partner with corporate risk teams to establish a repeatable, modular approach to facilitate model validation efforts and to increase efficiency in the process
• Identify, gather, retain, and publicize modeling artifacts required for approved and repeatable processes
• Collaborate with data scientists to understand business solutions, algorithms, and methodologies by use cases
• Responsible for model monitoring operations, reporting, and performance tracking activities
• Work with AI production teams and provide model monitoring business requirements
• May be called upon to review vendor models and solutions and/or models developed outside of EADS

Learn more about Wells Fargo / Analytics Career Video

Required Qualifications

  • 8+ years of experience in one or a combination of the following: reporting, analytics, or modeling
  • 4+ years of management experience
  • 2+ years of model governance experience
  • 2+ years of design, implementation and governance experience with Artificial Intelligence, Natural Language Processing or Machine Learning architecture
  • 2+ years of Natural Language Processing (NLP) experience
  • 2 + years of Model Validation Review experience

Desired Qualifications

  • A Master's degree or higher in a quantitative discipline
  • Ability to influence and build relationships with LOB stakeholders, technology CIO leadership, external service providers, and architecture teams
  • Ability to influence, partner, and negotiate with senior business leaders to gain commitment to accomplish business goals
  • Knowledge and understanding of Python, R or SCALA
  • Experience with Tensorflow, Theano or Keras

Other Desired Qualifications
  • 3+ years of experience with machine learning
  • Hands on experience with model validation, model monitoring, model review and model governance framework experience.
  • Experience working with big data infrastructure with tools such as Hive, Spark and h2o
  • Implementing solutions with common NLP frameworks and libraries in Python (NLTK, spaCy, gensim) or Java (Stanford CoreNLP, NLP4J)
  • Deep learning model architecture, applications or software (tensorflow, keras)
  • Partnership with audit and risk oversight functions experience


All offers for employment with Wells Fargo are contingent upon the candidate having successfully completed a criminal background check. Wells Fargo will consider qualified candidates with criminal histories in a manner consistent with the requirements of applicable local, state and Federal law, including Section 19 of the Federal Deposit Insurance Act.

Relevant military experience is considered for veterans and transitioning service men and women.

Wells Fargo is an Affirmative Action and Equal Opportunity Employer, Minority/Female/Disabled/Veteran/Gender Identity/Sexual Orientation.

Apply for this job


More Jobs Like This