Machine Learning 43 Rules: Best Practice Documents for Machine Learning (ML) Engineering

Google publishes best practice documents on machine learning (ML) engineering to help people who have mastered the basics of ML benefit from Google machine learning best practices. It introduces an ML style similar to the Google C++ style guide and other commonly used practical programming guides.

This document is designed to help people who have mastered the basics of ML to benefit from Google Machine Learning's best practices. It introduces an ML style similar to the Google C++ style guide and other commonly used practical programming guides. If you have studied ML courses, or have experience building or developing ML models, you have the necessary background knowledge to read this document.

the term

When we discuss effective ML, the following terms are repeatedly mentioned:

Example: Things to predict. For example, an instance can be a web page that you want to classify as "cat-related" or "not cat-related."

Tags: The answer to the forecasting task. It can be the answer generated by the ML system or the correct answer provided in the training data. For example, the label of a web page may be "related to cats."

Features: Properties of the instance used in the prediction task. For example, a web page might have the feature "contains the word 'cat'".

Feature Column: A group of related features, such as a collection of all countries where the user may live. The feature column of the sample may contain one or more features. "Feature column" is a Google-specific term. Features listed in the VM system used by Yahoo/Microsoft are called "namespaces" or fields.

Sample: An instance (and its characteristics) and a label.

Model: Statistical representation of the forecasting task. You use a sample to train a model and then use that model to make predictions.

Indicator: A value you care about. Perhaps (but not necessarily) can be directly optimized.

Goal: An indicator of which the algorithm is trying to optimize.

Pipeline: The infrastructure of the ML algorithm. Pipelines include collecting data from the front end, putting data into training data files, training one or more models, and applying the model to the production environment.

Clickthrough Rate: The percentage of visitors who clicked on the link in the ad.

Overview

To create a quality product:

Think of yourself as an outstanding engineer, not an ML expert.

In fact, most of the problems you will face are engineering problems. Even with all the resources that a brilliant ML expert has mastered, most of the gains are brought by suitable features (rather than precise machine learning algorithms). So, the basic way to do ML is:

Make sure the pipeline is solid and reliable from beginning to end.

Start by setting a reasonable goal.

Add commonsense features in a simple way.

Make sure the pipeline is always solid and reliable.

The above method will achieve good results in a long time. As long as you can still make progress with some simple techniques, you should not deviate from the above methods. Increasing complexity will slow down the release of future releases.

When you make full use of all the simple techniques, it may be time to explore ML's cutting-edge technologies. See the third phase of the "ML Project" section.

The structure of this document is as follows:

The first part can help you understand if the time to build an ML system is mature.

The second part explains how to deploy the first pipe.

The third section describes how to publish and iterate when adding new features to the pipeline, how to evaluate the model, and how to deal with training-application bias.

The last section explains what to do when you reach the stabilization phase.

This is followed by a list of relevant resources and appendices. The appendix provides some background information on the systems that are mentioned in this document as examples for many times.

Before ML

Rule 1: Do not be afraid to post products that do not use ML technology.

ML technology is cool, but it requires data. In theory, you can use data from other problems and then adjust the model for new products, but the effect is likely to be less than the basic heuristic algorithm. If you think that ML technology can bring you 100% improvement, then the heuristic algorithm can bring you a 50% increase.

For example, if you want to rank apps in the app market, you can use install rates or install times as heuristic algorithm metrics. If you want to detect spam, you can filter out publishers who have previously sent spam. Also, don't be afraid to modify it manually. If you need to rank contacts, they can be sorted (or sorted alphabetically) near and far in the order they are used. If your product does not have to use ML technology, do not use it until you have enough data.

Rule 2: First design and implement indicators.

Before formalizing the function of the ML system, the value of the indicator is tracked in the current system as much as possible. The reason for this is as follows:

Taking action in advance will make it easier to obtain authorization from users of the system.

If you think that you may need to consider an aspect in the future, it's best to start collecting relevant historical data immediately.

If you consider index measurements when designing your system, you will save a lot of effort in the future. Specifically, you don't want to find strings in your logs to measure metrics later!

You will find what has changed and what hasn’t changed. For example, suppose you want to directly optimize the number of single-day active users. However, during the early manipulation of the system, you may find that a significant change in the user experience does not significantly change the indicator.

The Google+ team measures the number of expansions, forwards, +1s, comments, and comment forwarding times per user for each reading, then uses this data to measure the quality of posts when the model is applied. Also note that the experimental framework is very important. You must group users into buckets in the experimental framework and summarize statistics by experiment. See rule 12.

By collecting metrics in a more liberal way, you can understand your system more fully. Found a problem? Add metrics to track it! Are you excited about some of the volume changes that occurred in the previous version? Add metrics to track it!

Rule 3: Select ML technology rather than sophisticated heuristics.

Simple heuristics are good for launching products. However, complicated heuristic algorithms are difficult to maintain. When you have enough data and basically determine what you are trying to achieve, consider using ML technology. As with most software engineering tasks, you need to constantly update methods (whether they are heuristics or ML models) and you will find that ML models are easier to update and maintain (see Rule 16).

ML Stage 1: Your First Pipeline

Focus on the system infrastructure of the first pipeline. Although it is interesting to look forward to all aspects of the innovative ML that you are about to undertake, if you do not first confirm the reliability of the pipeline, it is difficult to figure out what happened.

Rule 4: Ensure that the first model is easy to use and properly implement the infrastructure.

The first model can improve your product quality most effectively, so it doesn't need to be fancy and easy to use. However, you will encounter many unexpected infrastructure issues. Before you publicly launch a new ML system that you have carefully constructed, you must determine the following:

How to get a sample for your learning algorithm.

Initially determine what is the definition of "good" and "bad" for your system.

How to integrate the model into the application. You can apply the model online or you can use the sample offline to precompute the model and store the result in a table. For example, you may need to pre-categorize web pages and store the results in a table, but you may also want to classify chat messages online.

Selecting simple features makes it easier to ensure:

Apply these features correctly to your learning algorithm.

The model learns reasonable weights.

Apply these features to the server side correctly.

When you have a system that can reliably accomplish the above three points, you have completed most of the work. Simple models can provide you with benchmark metrics and benchmark behaviors that you can use to test more complex models. Some teams use "neutral" as the goal of the first release - clearly dilute machine learning outcomes at the time of initial release to avoid distractions.

Rule 5: Separate the machine learning and test the infrastructure separately.

Make sure that the infrastructure is testable and that the learning part of the system is encapsulated to test everything outside these parts. in particular:

Test the effect of the data import algorithm. Check if the feature column that should be filled is filled. With privacy permission, manually check the data entered into the training algorithm. If possible, look at the statistics in the pipeline and compare it with the statistics of the same data that was processed elsewhere.

The test derives the effect of the model from the training algorithm. Make sure that the model in the training environment gives the same score as the model in the application environment (see rule 37).

ML is unpredictable, so have tests for the code that creates the sample in the training environment and the application environment; and make sure that you can load and use the fixed model during the application. In addition, understanding your data is critical: See Practical advice for analyzing large, complex data sets.

Rule 6: Note the discarded data when copying pipes.

Normally, we create new pipes (ie, cargo worship programming) by duplicating existing pipes, and old pipes discard some of the data needed for new pipes. For example, the pipeline for Google+ Top Messages discards older posts (because it constantly tries to rank the latest posts). This pipe is duplicated for Google+ traffic. In messages, older posts still make sense, but old pipes still discard them. Another common mode is to record only the data that the user sees. Therefore, if we want to model the reason why users cannot see a particular post, such data is useless because the pipeline has discarded all of the negative classification samples. There have been similar problems in Play. While processing the Play app homepage, a new pipeline was created, which also contains samples from the Play Games landing page, but no features distinguish the source of each sample.

Rule 7: Turn heuristic algorithms into features or handle them externally.

In general, the problem ML tries to solve is not entirely new. There is an existing system that can be used to rank, sort, or resolve any issues that you are trying to solve. This means that there are many rules and heuristics. After using ML to make adjustments, this type of heuristic algorithm can provide you with convenience. You should mine your own heuristics to understand any information they contain for the following two reasons. First, the transition to the ML system will be smoother. Second, these rules often contain a large amount of intuition about the system that you do not want to discard.

You can use existing heuristics in four ways:

Use heuristics for preprocessing. If the feature is very good, you can choose to do this. For example, in the spam filter, if the sender is blacklisted, do not try to relearn what the "blacklisted" means. Block this email. This method is best suited for use in binary classification tasks.

Create features. Creating features directly through heuristics is a good practice. For example, if you use a heuristic algorithm to calculate the relevance score for a query result, you can include this score as a feature's value. You may want to use ML techniques later to adjust the value (for example, convert the value to one of a limited set of discrete values, or combine it with other features), but first use the original values ​​generated by the heuristic algorithm.

Mining the heuristic algorithm's original input. If an application heuristic algorithm combines the number of installs, the number of characters in the text, and the day of the week, consider splitting the content and providing it as input to the learning algorithm. Some of the techniques that apply to ensemble learning also apply here (see rule 40).

Modify the tag. You can choose to do this when you feel that the heuristic algorithm will get information that is not included in the current tab. For example, if you are trying to maximize the number of downloads, but you also want good content, the possible solution is to multiply the average number of stars obtained by the application with a tag. You have great flexibility in modifying tags. See "Your first goal."

When using heuristics in an ML system, be sure to watch for additional complexity. Using the old heuristic algorithm in the new ML algorithm can help achieve a smooth transition, but think about whether there is an easier way to achieve the same effect.

monitor

In general, implement good alarm security mechanisms, such as designing steps for resolving alerts and providing an "Information Center" page.

Rule 8: Understand the freshness of your system.

If you use the model one day ago, how much will the effect be reduced? What about the model a week ago? A quarter ago model? This type of message helps you understand what needs to be monitored first. If you do not update the model one day, it will have a serious impact on the quality of your product, it is best to let engineers continue to observe the situation. Most ad serving systems handle new ads every day and must be updated daily. For example, if you do not update the ML model of Google Play Search, it will have a negative impact in less than a month. There are no post identifiers in some models of Google+ Top Messages, so there is no need to export these models often. Other models with post identifiers update much more frequently. Also note that the freshness will change over time, especially when adding or removing feature columns from the model.

Rule 9: Detect the problem first and then export the model.

Many ML systems experience the stage of exporting the model to apply the model. If there is a problem with the exported model, it is a user-oriented issue.

Before exporting the model, check for soundness. Specifically, make sure that the model is reasonable in dealing with reserved data. Or, if you always think there is a problem with the data, don't export the model. Many teams deploying models often check the area under the ROC curve (abbreviated AUC) before exporting the model. When there is a problem with a model that has not yet been exported, an e-mail alert needs to be sent; however, when a user-facing model encounters a problem, it may need to be announced through a page. Therefore, it is best to wait until the inspection is complete and ensure that the model is completely safe before exporting the model to avoid impact on the user.

Rule 10: Pay attention to hidden issues.

This problem is more common with ML systems than with other types of systems. Assuming that the associated specific table is no longer updated, then the ML system will adjust accordingly and its behavior will still be quite good, but it will gradually become worse. Sometimes, you will find that some forms have not been updated in a few months, just refresh it, you can get more effective improvement than all the other improvements made in the corresponding quarter! The coverage of a feature may change as a result of a change: for example, a feature column may be filled in 90% of the sample, but the rate suddenly drops to 60%. Google Play had a table of outdated 6 months, but only refreshed the table, the installation rate increased by 2%. If you track the statistics of your data and manually check the data from time to time, you can reduce this failure.

Rule 11: Provide the owner of the feature column and related documents.

If the system is large and there are many feature columns, you need to know the creator or maintainer of each feature column. If you find that someone who understands a particular feature column is leaving, make sure someone knows the relevant information. Although many feature columns have descriptive names, it is a good idea to provide more detailed descriptions of the meaning of the features, the source, and the way in which they are expected to provide assistance.

Your first goal

You will pay attention to a lot of metrics or measurement results about the system, but you can usually only specify one goal for your ML algorithm, which is your algorithm's "attempted" optimization value. Here, I introduce the difference between goals and indicators: indicators are any number reported by your system and may or may not be important. See also rule 2.

Rule 12: When choosing which goal to optimize directly, do not think too much.

You want to make money, want customer satisfaction, want to make the world a better place. There are a lot of metrics you care about, and you should measure all of them (see Rule 2). However, in the early ML process, you will find that these indicators are on the rise, even those that you did not choose to optimize directly. For example, suppose you are concerned about the number of clicks and how long users stay on the site. If you optimize your clicks, it’s likely that your users’ time on the site will increase.

So, when you can still easily add all the metrics, keep it simple and don't worry too much about how to balance different metrics. However, do not overuse this rule: Do not confuse your goals with the system's final health (see rule 39). In addition, if you find yourself increasing your metrics for direct optimization but decide not to publish your system, you may need to modify some of your goals.

Rule 13: Choose a simple metric that is observable and attributable to your first goal.

You often don't know what the real goal is. You think you know it, but when you look at the data and compare the old system to the new ML system, you find that you want to adjust your goals. In addition, different members of the team usually cannot agree on what is the true goal. The ML goal should be a goal that satisfies the following criteria: an agent that is easy to measure and is a "real" goal. In fact, there is usually no "real" goal (see rule 39). Therefore, please train on simple ML goals and consider adding a "strategic layer" at the top so that you can add other logic (preferably very simple logic) for final ranking.

To model, the simplest indicator is user behavior that is directly observable and attributable to system operations:

Did the user click on this ranked link?

Has the user downloaded this ranked object?

Does the user forward/reply/email this ranked audience?

Did the user rate this ranked audience?

Does the user mark this displayed object as spam/sexual/aggressive content?

Avoid modeling the indirect effects at the beginning:

Did the user visit the site the next day?

How long have users stayed on the site?

How many active daily users?

In fact, indirect influences can be excellent indicators that can be used during A/B testing and release decisions.

Finally, don't try to make the ML system clear about the following issues:

Are users satisfied when using the product?

Is the user happy with the experience?

Does the product improve the overall satisfaction of the user?

How does this affect the overall operation of the company?

All of these are important, but they are also extremely difficult to measure. Use proxy metrics instead: If users are satisfied, they will stay longer on the site. If users are satisfied, they will visit the site again tomorrow. In terms of satisfaction and company operating conditions, human judgment is required to correlate any ML goals with the nature and business plan of the products you sell.

Rule 14: Start with an interpretable model to make debugging easier.

Linear regression, logistic regression, and Poisson regression are directly driven by the probability model. Each forecast can be viewed as a probability or expected value. In this way, the model is easier to debug than models that use the target (0-1 loss, various hinge loss functions, etc.) to try to directly optimize the classification accuracy or rank the effect. For example, there is a problem if there is a discrepancy between the probabilities obtained in training and the probabilities predicted by using side-by-side analysis or by examining the production system.

For example, in Linear Regression, Logistic Regression, or Poisson Regression, there is a portion of the average forecasted expected value equal to the average tag value (first-moment calibration, or just calibration). Assuming you have no regularization and the algorithm has converged, then this is theoretically the case, and in fact this is almost the case. If you have a feature, for each sample whose value is either 0 or 1, the 3 sample sets with feature value 1 are calibrated. In addition, if you have a feature that has a value of 1 for each sample, all sample sets will be calibrated.

With a simple model, you can more easily handle feedback loops (see rule 36). In general, we make decisions based on these probabilistic forecasts; for example, rank the posts in descending order based on the expected values ​​(click probability/download probability, etc.). However, please note that when choosing the model to use, your decision is more important than the probability given by the model (see rule 27).

Rule 15: Distinguish junk content filtering and quality ranking in the policy layer.

Quality ranking is an art, but spam filtering is like a war. For users who use your system, the signals you use to determine high quality posts will become apparent, and these users will adjust their posts to have the attributes of high quality posts. Therefore, your quality ranking should focus on ranking honestly published content. You should not apply discounts to spam content because the Quality Ranking Learner ranks spam. Similarly, content that is "not suitable for children" should not be dealt with in the quality ranking. Spam filtering is another matter. You must understand that the characteristics that need to be generated will constantly change. Normally, you will set some obvious rules in the system (if a post has received more than three spam reports, do not retrieve the message, etc.). All learning models must be updated at least daily. Content creators' reputation will play a big role.

At a certain level, the output of these two systems must be integrated together. Please note that when filtering spam in search results, it may be more proactive than filtering spam. The premise of this statement is that you have no regularization and the algorithm has converged. This is generally the case. In addition, it is a standard practice to remove spam from the training data of the mass classifier.

ML Second Stage: Feature Engineering

In the first phase of the ML system life cycle, the important issues involved three aspects: the introduction of training data into the learning system, measurement of any indicators of interest, and the construction of application infrastructure. After you have built an end-to-end stable operation system and have developed system tests and unit tests, you can enter the second stage.

Many of the goals of the second phase are easy to implement, and there are many obvious features that can be introduced into the system. Therefore, the second phase of ML involves introducing as many features as possible and combining them in an intuitive way. At this stage, all the indicators should still be on the rise, you will release the system several times, and it is very suitable for arranging multiple engineers to integrate all the data needed to create truly outstanding learning systems.

Rule 16: Develop a release and iterative model plan.

Don't expect that the model you are building now will be the last model you are about to publish, and don't expect you to stop publishing models. Therefore, please consider whether the increased complexity in this release will slow the release of future releases. Many teams release one or more models every quarter over the years. The three basic reasons for publishing a new model are as follows:

You are about to add new features.

You will adjust the regularization and combine the old features in new ways.

You will adjust the goal.

In any case, there are no disadvantages to consider when building a model: Viewing the data provided to the sample helps to discover new signals, old signals, and damaged signals. Therefore, when building a model, consider how easy it is to add, remove, or recombine features. Consider creating a fresh copy of the pipeline and how easy it is to verify its correctness. Consider whether you can run two or three copies at the same time. Finally, there is no need to worry about whether this version of the pipeline includes the 16th feature (35 in total), which will be included in the next quarter.

Rule 17: Start with characteristics that can be observed and reported directly (rather than learned characteristics).

This point may be controversial, but many problems can be avoided. First of all, let's introduce what is the characteristics of learning. Features learned are features that are generated by external systems (such as unsupervised cluster systems) or the learners themselves (such as through factor models or deep learning). The features generated by these two methods are very useful, but they cause many problems and should not be used in the first model.

If you use external systems to create features, be aware that external systems have their own goals. There may be only a little correlation between the goals of the external system and your current goals. If you acquire a transient state of an external system, it may expire. If you update features from an external system, the meaning of the features may change. If you use external systems to provide features, please note that this method requires great care.

The main problem with the factor model and the depth model is that they are non-convex models. Therefore, there is no guarantee that the optimal solution can be simulated or found, and the local minima found at each iteration may be different. This change makes it difficult to determine whether the impact of a change in the system is meaningful or random. By creating a model with no depth features, you can get excellent benchmark results. After reaching this baseline, you can try a more esoteric approach.

Rule 18: Explore the characteristics of content that can be generalized across contexts.

ML systems are usually just a small part of a larger system. For example, imagine posts that are likely to be used in popular messages. Many users have +1'd, forwarded, or commented on popular posts before they were displayed. If you provide these statistics to the learner, it will promote new posts that have no data in the optimized scenario. YouTube's "Next Watch" can use YouTube Views or Views (Views of another video after watching one video) or explicit user ratings to recommend content. Finally, if you use a user action as a tag, seeing the user perform the action on the document in other situations can be a good feature. With all of these features, you can introduce new content into the situation. Please note that this has nothing to do with personalization: first find out if someone likes the content in this situation and then figure out how much to like.

Rule 19: Use very specific characteristics wherever possible.

For massive data, learning millions of simple features is easier than learning a few complex features. The identifier of the document being retrieved and the normalized query will not provide much generalization, but it will keep your ranking consistent with the tags of the most frequently used queries. Therefore, don't be afraid of feature groups that have the following characteristics: Each feature applies to a small portion of your data but overall coverage is above 90%. You can use regularization to eliminate features that apply too few samples.

Rule 20: Combine and modify existing features to create new features in an easy-to-understand way.

There are many ways to combine and modify features. With an ML system such as TensorFlow, you can preprocess the data through transformations. The two most standard methods are "discretization" and "combination."

"Discretization" refers to extracting a continuous feature and creating many discrete features from it. Take the continuous feature of age as an example. You can create a feature with a value of 1 when you are under 18 and create another feature with a value of 1 between ages 18-35, and so on. Don't worry too much about the boundaries of these histograms: Basic quantiles have the greatest impact on you.

The "combination" method refers to the combination of two or more feature columns. In TensorFlow, characteristic columns refer to homogeneous feature sets (for example, {men, women}, {United States, Canada, Mexico}, etc.). A combination refers to a new feature column that contains features such as {male, female} × {United States, Canada, Mexico}. This new feature column will contain features (male, Canada). If you are using TensorFlow and have TensorFlow create this combination for you, this (male, Canadian) feature will exist in a sample that represents a Canadian male. Note that you need to have a lot of data to use a combined learning model with three, four, or more datum feature columns.

Combinations that generate very large feature columns may overfit. For example, suppose you are performing some sort of search. One of your characteristic columns contains the words in the query and the other one contains the words in the document. At this point, you can combine these feature columns using the "combination" method, but you end up with many features (see rule 21).

There are two alternative methods for processing text. The most demanding method is dot product. When the dot product method uses the simplest form, only the number of common words between the query and the document is calculated. Then this feature is discretized. Another method is intersection: if the intersection method is used, a feature will appear if and only if both the document and the query contain the word "pony"; if and only if both the document and the query contain the word "the" Only another feature will appear.

Rule 21: The number of feature weights you can learn in a linear model is roughly proportional to the amount of data you have.

There are various excellent results of statistical learning theory on the appropriate complexity of the model, but you basically only need to understand this rule. In a conversation, people have expressed such doubts: Can we learn anything from a thousand samples, or if we need more than one million samples, they have such doubts because they are limited to A specific learning style. The key is to adjust your learning model based on the data size:

If you are building a search ranking system with millions of different words in documents and queries, and you have 1000 labeled samples, you should manually engineer in document and query features, TF-IDF, and multiple other heights There is a dot product between the characteristics of the chemical. You have 1000 samples and more than a dozen features.

If you have one million samples, use regularization and feature selection (possibly) to intersect the document feature column with the query feature column. As a result, you will get millions of features; but if you use regularization, you get fewer features. You will have tens of millions of samples and may produce 100,000 features.

If you have billions or hundreds of billions of samples, you can use feature selection and regularization to combine feature columns with document and query tags. You will have one billion samples and ten million features. Statistical learning theory rarely sets strict limits, but it can provide a good starting point for guidance.

Finally, according to Rule 28 decide which features to use.

Rule 22: Clean up features that are no longer used.

Unused features generate technical liabilities. If you find yourself not using a feature, and combining it with other features does not work, then remove it from your infrastructure. You need to keep your infrastructure simple so that you try the features that are most likely to bring good results as quickly as possible. If necessary, Others can add your characteristics back at any time.

When determining which features to add or retain, consider coverage. That is, how many samples are covered by the corresponding features? For example, if you have some personalization features, but only 8% of users have personalization features, the effect will not be very good.

At the same time, some features may exceed their weight. For example, if one of your characteristics covers only 1% of the data, but 90% of the samples with that characteristic are positive classifications, this is a good feature that can be added.

Manual analysis of the system

Before continuing to explore the third phase of ML, it is important to focus on what you cannot learn in any ML course: how to examine existing models and make improvements. This is more like an art than science, but there are several anti-patterns that need to be avoided.

Rule 23: You are not a typical end user.

This may be the easiest way to get the team in trouble. Although there are many advantages to fishfood (using prototypes within the team) and dogfood (using prototypes within the company), employees should see if they meet the performance requirements. Although you should avoid applying significantly worse changes, when it comes to production, any changes that seem reasonable should be further tested, in two ways: ask non-professionals to answer paid questions on a crowdsourcing platform, or Users conduct online experiments.

There are two reasons for this. First of all, your relationship with the code is too close. You may be concerned with a particular aspect of the post, or you may just be putting too much emotion into it (such as confirming deviations). Second, your time is precious. Consider how many of the nine engineer's expenses spent on an hour-long meeting can buy on the crowdsourcing platform.

If you really want to get user feedback, use the user experience method.在流程的早期阶段创建用户角色（请参阅比尔- 布克斯顿的Sketching User Experiences 一书中的描述），然后进行可用性测试（请参阅史蒂夫- 克鲁格的Don't Make Me Think 一书中的描述）。用户角色是指创建假想用户。例如，如果您的团队成员都是男性，则有必要设计一个35 岁的女性用户角色（使用用户特征完成），并查看其生成的结果，而不是只查看10 位25-40 岁男性的结果。在可用性测试中请真实用户体验您的网站（通过本地或远程方式）并观察他们的反应也可以让您以全新的视角看待问题。

第24 条规则：衡量模型间的差异。

在向任何用户展示您的新模型之前，您可以进行的最简单（有时也是最有用）的一项衡量是，评估新模型的结果与生产有多大差别。例如，如果您有一项排名任务，则在整个系统中针对一批示例查询运行这两个模型，并查看结果的对称差分有多大（按排名位置加权）。如果差分非常小，那么您无需运行实验，就可以判断不会出现很大变化。如果差分很大，那么您需要确保这种更改可以带来好的结果。查看对称差分较大的查询有助于您了解更改的性质。不过，请确保您的系统是稳定的。确保模型与自身之间的对称差分较低（理想情况下为零）。

第25 条规则：选择模型时，实用效果比预测能力更重要。

您的模型可能会尝试预测点击率。但归根到底，关键问题在于您用这种预测做什么。如果您使用该预测对文档进行排名，那么最终排名的质量比预测本身更重要。如果您要预测一个文档是垃圾内容的概率，然后选择一个取舍点来确定要阻断的内容，那么允许的内容的精确率更为重要。大多数情况下，这两项应该是一致的：当它们不一致时，带来的优势可能会非常小。因此，如果某种更改可以改善对数损失，但会降低系统的性能，则查找其他特征。当这种情况开始频繁发生时，说明您该重新审视模型的目标了。

第26 条规则：在衡量的错误中寻找规律，并创建新特征。

假设您看到模型"弄错" 了一个训练样本。在分类任务中，这种错误可能是假正例，也可能是假负例。在排名任务中，这种错误可能是假正例和假负例，其中正例的排名比负例的排名低。最重要的是，ML系统知道自己弄错了该样本，如果有机会，它会修复该错误。如果您向该模型提供一个允许其修正错误的特征，该模型会尝试使用它。

另一方面，如果您尝试根据系统不会视为错误的样本创建一个特征，该特征将会被系统忽略。例如，假设用户在Play 应用搜索中搜索"免费游戏"。假设排名靠前的搜索结果中有一个是相关性较低的搞笑应用。因此，您为"搞笑应用" 创建了一个特征。但是，如果您要最大限度地增加安装次数，并且用户在搜索免费游戏时安装了搞笑应用，那么"搞笑应用" 特征不会达到您想要的效果。

如果模型弄错了您的某些样本，请在当前特征集之外寻找规律。例如，如果系统似乎在降低内容较长的帖子的排名，那么添加帖子长度。不要添加过于具体的特征。如果您要添加帖子长度，请不要试图猜测长度的具体含义，只需添加十多个特征，然后让模型自行处理（请参阅第二十一条规则）。这是实现目标最简单的方式。

第27 条规则：尝试量化观察到的异常行为。

当现有的损失函数没有捕获您团队中的部分成员不喜欢的某些系统属性时，他们会开始有挫败感。此时，他们应该竭尽所能将抱怨转换成具体的数字。例如，如果他们认为Play 搜索中显示的"搞笑应用" 过多，则可以通过人工评分识别搞笑应用。（在这种情况下，您可以使用人工标记的数据，因为相对较少的一部分查询占了很大一部分流量。）如果您的问题是可衡量的，那么您可以开始将它们用作特征、目标或指标。一般规则是" 先量化，再优化 "。

第28 条规则：请注意，短期行为相同并不意味着长期行为也相同。

假设您的新系统会查看每个doc_id 和exact_query，然后计算每个查询的每个文档的点击概率。您发现在并排分析和A/B 测试中，其行为与您当前系统的行为几乎完全相同，考虑到它的简单性，您发布了它。不过，您发现它没有显示任何新应用。 why?那是因为您的系统仅根据自己的查询历史记录显示文档，所以不知道应该显示新文档。

了解这种系统长期行为的唯一方法是，仅使用模型在线时获得的数据对其进行训练。这一点非常难。

训练- 应用偏差

训练- 应用偏差是指训练效果与应用效果之间的差异。出现这种偏差的原因可能是：

训练管道和应用管道中数据的处理方式有差异。

训练时和应用时所用数据有变化。

模型和算法之间有反馈环。

我们注意到Google 的生产ML系统也存在训练- 应用偏差，这种偏差对性能产生了负面影响。最好的解决方案是明确进行监控，以避免在系统和数据改变时引入容易被忽视的偏差。

第29 条规则：确保训练效果和应用效果一样的最佳方法是，保存在应用时使用的特征集，然后将这些特征通过管道传输到日志，以便在训练时使用。

即使您不能对每个样本都这样做，也对一小部分样本这样做，以便验证应用和训练之间的一致性（请参阅第37 条规则）。采取了这项措施的Google 团队有时会对结果感到惊讶。 YouTube 首页改用这种在应用时记录特征的做法后，不仅大大提高了质量，而且减少了代码复杂度。目前有许多团队都已经在其基础设施上采用了这种方法。

第30 条规则：按重要性对采样数据加权，不要随意丢弃它们！

数据过多时，总会忍不住采用前面的文件而忽略后面的文件。这是错误的做法。尽管可以丢弃从未向用户展示过的数据，但对于其他数据来说，按重要性加权是最佳选择。按重要性加权意味着，如果您决定以30% 的概率对样本X 进行抽样，那么向其赋予10/3 的权重。按重要性加权时，您仍然可以使用第14 条规则中讨论的所有校准属性。

第31 条规则：如果您在训练和应用期间关联表格中的数据，请注意，表格中的数据可能会变化。

假设您将文档ID 与包含这些文档的特征（例如评论次数或点击次数）的表格相关联。表格中的特征在训练时和应用时可能有所不同。那么，您的模型在训练时和应用时对同一文档的预测就可能会不同。要避免这类问题，最简单的方法是在应用时记录特征（请参阅第32 条规则）。如果表格只是缓慢发生变化，那么您还可以每小时或每天创建表格快照，以获得非常接近的数据。请注意，这仍不能完全解决问题。

第32 条规则：尽可能在训练管道和应用管道间重复使用代码。

批处理不同于在线处理。进行在线处理时，您必须在每个请求到达时对其进行处理（例如，您必须为每个查询单独进行查找），而进行批处理时，您可以组合任务（例如进行关联）。应用时，您进行的是在线处理，而训练时，您进行的是批处理。不过，您可以通过一些方法来重复使用代码。例如，您可以专门为自己的系统创建一个对象，其中所有查询结果和关联都能以非常易于人类读取的方式进行存储，且错误也可以轻松进行测试。然后，收集了所有信息后，您可以在应用和训练期间使用一种共同的方法，在人类可读对象（特定于您的系统）和ML需要的任何格式之间架起一座桥梁。这样可以消除训练- 应用偏差的一个根源。由此推知，在训练和应用时，尽量不要使用两种不同的编程语言。如果这样做，就几乎不可能共享代码了。

第33 条规则：如果您根据1 月5 日之前的数据生成模型，则根据1 月6 日及之后的数据测试模型。

一般来说，要衡量模型的效果，应使用在训练模型所有数据对应的日期之后的日期收集的数据，因为这样能更好地反映系统应用到生产时的行为。如果您根据1 月5 日之前的数据生成模型，则根据1 月6 日及之后的数据测试模型。您一般会发现，使用新数据时模型的效果不如原来好，但应该不会太糟。由于可能存在的一些日常影响，您可能没有预测到平均点击率或转化率，但曲线下面积（表示正分类样本的分数高于负分类样本的概率）应该非常接近。

第34 条规则：在有关过滤的二元分类（例如，垃圾邮件检测或确定有趣的电子邮件）中，在短期内小小牺牲一下效果，以获得非常纯净的数据。

在过滤任务中，标记为负分类的样本不会向用户显示。假设您的过滤器在应用时可屏蔽75% 的负分类样本。您可能会希望从向用户显示的实例中提取额外的训练数据。例如，如果用户将您的过滤器未屏蔽的电子邮件标记为垃圾邮件，那么您可能想要从中学习规律。

但这种方法会引入采样偏差。如果您改为在应用期间将所有流量的1% 标记为"预留"，并向用户发送所有预留样本，则您可以收集更纯净的数据。现在，过滤器屏蔽了至少74% 的负分类样本。这些预留样本可以成为训练数据。

请注意，如果过滤器屏蔽了95% 或以上的负分类样本，则此方法的可行性会降低。即便如此，如果您希望衡量应用效果，可以进行更低比例的采样（比如0.1% 或0.001%）。一万个样本足以非常准确地评估效果。

第35 条规则：注意排名问题中存在的固有偏差。

当您彻底改变排名算法，导致出现不同的排名结果时，实际上改变了您的算法以后会处理的数据。这时，就会出现固有偏差，您应该围绕这种偏差来设计模型。具体方法有多种。以下是让您的模型青睐已见过的数据的方法。

对覆盖更多查询的特征（而不是仅覆盖一个查询的特征）进行更高的正则化。通过这种方式，模型将青睐专门针对一个或几个查询的特征，而不是泛化到所有查询的特征。这种方法有助于防止十分热门的查询结果显示到不相关的查询中。请注意，这与以下更为传统的建议相左：对具有更多唯一值的特征列进行更高的正则化。

仅允许特征具有正权重。这样一来，就可确保任何好特征都比"未知" 特征合适。

不选择只处理文档数据的特征。这是第一条规则的极端版本。例如，即使指定应用是热门下载应用（无论查询是什么），您也不想在所有地方都展示它。如果不选择只处理文档数据的特征，这一点很容易做到。您之所以不想在所有地方展示某个特定的热门应用，是因为让用户可以找到所有所需应用至关重要。例如，如果一位用户搜索"赏鸟应用"，他/她可能会下载"愤怒的小鸟"，但那绝对不是他/她想要的应用。展示此类应用可能会提高下载率，但最终却未能满足用户的需求。

第36 条规则：通过位置特征避免出现反馈环。

内容的位置会极大地影响用户与其互动的可能性。如果您将应用放在首位，则应用获得的点击率更高，导致您认为用户更有可能点击该应用。处理此类问题的一种方法是添加位置特征，即关于内容在网页中的位置的特征。您可以使用位置特征训练模型，使模型学习（例如）对特征"1stposition" 赋予较高的权重。因此，对于具有"1stposition=true" 特征的样本的其他因素，模型会赋予较低的权重。然后，在应用时，您不向任何实例提供位置特征，或为所有实例提供相同的默认特征，因为在决定以怎样的顺序显示候选实例之前，您就对其进行了打分。

请注意，因为训练和测试之间的这种不对称性，请务必在位置特征与模型的其余特征之间保持一定的分离性。让模型成为位置特征函数和其余特征函数之和是理想的状态。例如，不要将位置特征与任何文档特征组合在一起。

第37 条规则：测量训练/ 应用偏差。

一般来说，很多情况都会引起偏差。此外，您可以将其分为以下几个部分：

训练数据和预留数据的效果之间的差异。一般来说，这种情况始终存在，而且并非总是坏事。

预留数据和"次日" 数据的效果之间的差异。同样，这种情况始终存在。您应该调整正则化，以最大程度地提升次日数据的效果。不过，如果与预留数据相比，次日数据效果下降明显，则可能表明某些特征具有时效性，而且可能会降低模型的效果。

"次日" 数据和实时数据的效果之间的差异。如果您将模型应用于训练数据中的某个样本，并在应用时使用同一样本，那么您得到的结果应该完全相同（请参阅第5 条规则）。因此，此处的差异很可能表示出现了工程错误。

ML第三阶段：缓慢增长、优化细化和复杂模型

第二阶段即将结束时会出现一些信号。首先，月增长开始减弱。您将开始在指标之间做出取舍：在部分试验中，您会看到一些指标上升了，而另一些指标下降了。情况变得有趣起来。由于越来越难实现增长，因此ML系统必须变得更加复杂。注意：相比之前两个部分，本部分中会有较多的纯理论性规则。我们见过许多团队在ML的第一阶段和第二阶段非常满意。但到了第三阶段后，他们必须找到自己的道路。

第38 条规则：如果目标不协调，并成为问题，就不要在新特征上浪费时间。

当您的衡量结果稳定时，您的团队会开始关注当前ML系统的目标范围之外的问题。如前所述，如果现有算法目标未涵盖产品目标，则您需要修改算法目标或产品目标。例如，您可以优化点击次数、+1次数或下载次数，但让发布决策部分依赖于人工评分者。

第39 条规则：发布决策代表的是长期产品目标。

Alice 有一个关于减少预测安装次数的逻辑损失的想法。她添加了一个特征。逻辑损失降低了。当她运行在线实验时，看到安装率增加了。但是，在发布评审会上，有人指出，每日活跃用户数减少了5%。于是，团队决定不发布该模型。Alice 很失望，但现在她意识到发布决策取决于多个条件，只有一部分条件可以通过ML直接得到优化。

事实上，现实世界并不是网游世界：没有"生命值" 来确定产品的运行状况。团队必须使用自己收集的统计信息来尝试有效地预测系统未来的表现会如何。他们需要关注互动度、日活跃用户数(DAU)、30 日DAU、收入以及广告主的投资回报率。这些可在A/B 测试中衡量的指标本身仅代表了以下更长期目标：让用户满意、增加用户数量、让合作伙伴满意以及实现盈利，进一步，您还可以认为它们代表了发布优质且实用的产品，以及五年后公司繁荣发展。

唯一可以轻松做出发布决策的情况是，所有指标都在变好（或至少没有变差）。 如果团队能够在复杂的ML算法和简单的启发式算法之间做出选择，而对所有这些指标来说，简单的启发式算法可以提供更好的效果，那么应该选择启发式算法。此外，并未对所有可能的指标值进行明确排名。具体而言，请考虑以下两种情形：

实验 每日活跃用户数 收入/ 日

A	100 万	400 万美元
B	200 万	200 万美元

如果当前系统是A，那么团队不太可能会改用B。如果当前系统是B，那么团队不太可能会改用A。这似乎与理性行为背道而驰；但是，对更改指标的预测可能会成功也可能不会，因此这两种改变都蕴含着巨大的风险。每个指标都涵盖了团队所担心的一些风险。

此外，没有一个指标涵盖团队最关心的问题，即"五年后我的产品将何去何从"？

另一方面，个人更倾向于选择可以直接优化的目标。 大多数ML工具也都青睐这样的环境。在这样的环境下，快速创建新特征的工程师能稳定地进行一系列发布。一种称为"多目标学习" 的ML已开始解决此问题。例如，您可以提出约束满足问题，对每个指标设定下限，并优化指标的一些线性组合。不过，即使如此，也并不是所有指标都可以轻松框定为ML目标：如果用户点击了文档或安装了应用，那是因为相应内容展示出来了。但要弄清楚用户为什么访问您的网站就难得多。如何预测整个网站未来的成功状况属于AI完备问题：与计算机视觉或自然语言处理一样难。

第40 条规则：保证集成学习简单化。

采用原始特征并直接对内容进行排名的统一模型是最易于进行调试和理解的模型。但是，集成学习模型（将其他模型的分数结合到一起的模型）可以实现更好的效果。为了简单起见，每个模型应该要么是仅接受其他模型的输入的集成学习模型，要么是接受多个特征的基本模型，但不能两者皆是。 如果在单独训练的模型之上还有其他模型，则组合它们会导致不良行为。

使用简单的模型进行集成学习（仅将"基本" 模型的输出作为输入）。此外，您还需要将属性强加到这些集成学习模型上。例如，基本模型生成的分数的升高不应使集成学习模型的分数有所降低。另外，如果传入的模型在语义上可解释（例如，经过校准），则最理想，因为这样一来，即使基本模型发生改变，也不会扰乱集成学习模型。另外，强制要求：如果基本分类器的预测概率增大，不会使集成学习模型的预测概率降低。

第41 条规则：效果达到平稳后，寻找与现有信号有质的差别的新信息源并添加进来，而不是优化现有信号。

您添加了一些有关用户的受众特征信息，也添加了一些有关文档中字词的信息。您探索了模板，并调整了正则化。但在几个季度的发布中，关键指标的提升幅度从来没有超过1%。现在该怎么办？

是时候开始为截然不同的特征（例如，用户在过去一天内、一周内或一年内访问的文档的历史记录，或者其他属性的数据）构建基础架构了。您可以使用维基数据条目或公司内部信息（例如，Google 的知识图谱）。利用深度学习。开始调整您对投资回报的预期，并付出相应的努力。与在任何工程项目中一样，您必须对添加新特征的好处与增加复杂性的成本进行一番权衡。

第42 条规则：不要期望多样性、个性化或相关性与热门程度之间的联系有您认为的那样密切。

一组内容中的多样性可以有多种含义，其中内容来源的多样性是最常见的一种。个性化意味着每个用户获得贴合其个人需求的结果。相关性意味着某个特定查询的结果更适合该查询，而非其他任何查询。因此，这三个属性均具有不同于常态的定义。

但常态往往很难被打败。

请注意，如果您的系统在测量点击次数、访问时间、观看次数、+1 次数、转发次数等数据，那么您测量的是内容的热门程度。团队有时会尝试学习具备多样性的个性化模型。为实现个性化，他们会添加支持系统进行个性化（代表用户兴趣的部分特征）或多样化（表明相应文档是否与其他返回的文档有任何相同特征的特征，例如作者或内容）的特征，然后发现这些特征的权重比预期低（或者有时是不同的信号）。

这并不意味着多样性、个性化或相关性不重要。正如上一条规则中所指出的那样，您可以进行后期处理来增加多样性或相关性。如果您看到更长期的目标有所增长，您可以声明除了热门程度外，多样性/相关性也很有价值。然后，您可以继续采用后期处理方法，也可以根据多样性或相关性直接修改目标。

第43 条规则：在不同的产品中，您的好友基本保持不变，但您的兴趣并非如此。

Google 的团队通过以下做法取得了大量进展：采用一个预测产品中某种联系的紧密程度的模型，并使用该模型对其他产品进行准确预测。您的好友保持不变。另一方面，我曾见过几个团队在应对多个产品间的个性化特征时捉襟见肘。是的，当时看起来应该可以奏效的。但现在看来并没有。有时可以奏效的方法是，使用一个属性的原始数据来预测另一个属性的行为。此外，请注意，仅仅是知道用户有其他属性的历史记录也会有帮助。例如，两个产品上出现了用户活动或许本身就可以说明该问题。

Battery Holders

Battery holders are key components in many types of portable and anchored electrical items - and you`ll find just as many varieties, brands, sizes and types of battery holder available across global markets as there are different sorts of battery.

In this guide, we`ll look more closely at some of the various mounting styles a battery holder might come fitted with, as well as the many different cell sizes they`re designed to work with.

What is a battery holder?
Battery Holders & Mounts
A battery holder is most commonly sold as an integral or removable compartment or cavity, designed to be inserted into - or attached onto - a suitable item of cell-powered equipment.

The primary function of a battery holder is to keep cells fixed in place safely and securely while conveying power from the batteries to the device in question. External connections on battery holders are most often made by contacts either with pins, surface mount feet, soldered lugs or via a set of wire leads.

Battery holders are often designed to be incorporated within the body of an electrical item, but they`re also frequently sold as external compartments or attachments. Either way, some of the most important factors to consider when choosing an appropriate battery holder for a given application will be:

Battery size
Compatible cell types
Contact/terminal style
Battery mounting and insertion method
Unit size and shape
Battery holder cover and housing design
Build quality and manufacturer reputation

Holders and mounts for different battery sizes
Many different battery holder sizes and mounting types are available from manufacturers and suppliers all over the world. Most will be made to work with a particular sort of cell - or sometimes with a range of multiple battery sizes - and to be affixed to the electrical item in question in a specific way.

Bear in mind that, in addition to sorting through the wide variety of battery holder sizes and capacities on the market, you`ll also need to be mindful of the many options available in terms of actual battery chemistry.

While the vast majority of holders and mounts will be able to accommodate most battery chemistry types, each will have its own unique set of pros and cons in terms of performance, power drain, longevity, cost and environmental impact.

Common battery types sold by many suppliers might include:

Alkaline batteries
Lithium batteries
Rechargeable batteries (including li-ion)
Nickel cadmium batteries
Nickel metal hydride batteries
NiZN batteries
The above range of cell types is often available across a broad range of popular battery sizes and shapes, including both AA and AAA versions. Most, but not all, will be interchangeable in a wide range of devices, holders and mounts. However, it`s seldom advisable to combine mixed battery chemistries in a single electronic device.

Doing so can often impede device performance, as the item drawing power will only ever perform to the capabilities of the weakest battery in the holder. Of even greater concern is the fact that more powerful batteries can sometimes force weaker ones to work harder than they`re intended to, causing them to drain to very low voltage levels and possibly leading to potential leaking or rupture.

As such, it`s always wise to ensure you know which cell types you`re likely to be using in advance of making a purchase, as well as the correct sizing you`ll require.

AA Battery Holders & Mounts

An AA size battery holder is one of the more common types of battery storage compartments you`ll see on sale from UK suppliers. AA size (`double-A`), is what most people would typically think of as a `standard` battery, and remains extremely common in all kinds of portable electrical items and equipment.

AA battery pack holders can be manufactured to house any number of cells in theory, but in daily use it`s usually anywhere between 1-10 batteries, with 2-8 being by far the most popular options. AA single battery holders are also common.

Relatively few cell-powered tools or other pieces of electrical equipment will be designed to require more than 8 AA batteries for achieving sufficient power supply. Items that demand a higher power draw than this will usually be built around a larger cell type, for improved practicality and less frequent need for battery replacement or recharging.

AAA Battery Holders and Mounts

Battery holders for AAA cells, commonly referred to as `triple-A` are another very common configuration. AAA batteries are smaller and slimmer than AA versions, with around a third of the storage capacity.

Due to the increasing efficiency of many device types, AAA batteries are often found in more compact pieces of equipment that would typically have been expected to run off double-A cells in years gone by. Today, these will very often include low-drain portable or handheld electronic goods or tools, such as:

Remote controls
Mp3 players, handheld gaming systems and other portable entertainment devices
Digital cameras
Cordless mice or keyboards
Flashlights and other portable lighting units
Clocks and alarms
Kitchen gadgets
AAA battery holders are generally built to accommodate fewer individual cells than bigger units for housing AA and larger capacity batteries, as devices requiring more power will typically not run off triple-A cells.

An AAAA battery holder is a somewhat less common configuration, given that the much smaller than average AAAA (`quadruple-A` or `four-A`) cell standard drains considerably faster than AA or AAA equivalents.


C Battery Holders and Mounts

C size battery holders are designed around the fairly common C battery (sometimes known as an R14). This is a fairly wide diameter cylindrical cell, smaller than D batteries while being the same height - but significantly wider - than AA standard.

C cell batteries, holders and mounts are less widely used than either AA or D versions, particularly since power efficiency is continually improving across most cell-powered devices. However, they can still often be found on a range of medium-drain electronic items, including:

Heavier flashlights and smaller ambient lighting fixtures (e.g. table lamps)
Radios and portable speakers
Toys and games Musical instruments and amplifiers

D Battery Holders and Mounts

D batteries are among the larger types of `everyday use` cells you`ll commonly encounter around the home or workplace. D size batteries tend to be used for higher-drain devices that tend to be used for longer periods at a time, including:

High Power torches and flashlights
Safety systems
Geiger counters
Megaphones, larger musical instruments and more powerful audio devices or speakers
Motorised toys and portable gaming consoles
Products with inbuilt electric motors
Battery holders for D cells are most commonly sold in configurations designed to house either 2 or 4 individual batteries.

12V Battery Holders and Mounts

Battery holders for 12V cells are somewhat less widely available, chiefly because the 12V standard incorporates such a wide range of battery shapes and sizes, although 12V Battery Contacts themselves are more commonly sold as separate components (see Battery holder mounts, below).

The precise sort of 12V battery you might need for specific applications will dictate the appropriate form factor, which can be anything from a large heavy car battery type to one closely resembling a traditional AA cell.

With larger 12V batteries, in particular, it`s less common to need more than one at a time, and they`re generally built more intrinsically into the device or equipment being powered than other types of `snap in/out` batteries designed for frequent and quick replacement. This means that 12V battery holders and mounts aren`t as frequently sold by many UK suppliers as versions for other, more common cell types.

Battery holder mounts
Battery holder mounts usually refer to the way the battery holder itself is slotted, secured or attached onto or into the main body of the item being powered. Note, however, that the term `battery mount` is also used to refer to the method by which individual cells are held in place within the overall holder unit.

Battery holder mounts and battery mounts are usually present as an integral part of a complete holder unit, but you can also buy battery mounts cheaply as swap-out or replacement parts. They can come in many different configurations and designs, with some of the more common types being:

Chassis mount
Panel mount
PCB mount
Slide-in mount (usually refers to the battery insertion method)
Snap-in mount (usually refers to the battery insertion method)
Strap & lead mount (a battery connector, rather than a holder per se)
Surface mount
Through-hole mount
Wire lead mount (refers to connection type)
In this section, we`ll look at some of the most widely used battery mount types in everyday use

Chassis Mount Battery Holders

A chassis mount battery holder denotes one intended to be affixed to a surface plate within the main body of the device being powered, and will usually remain in place while batteries are being swapped. A chassis mount holder is usually attached either via fixing holes in the back of the casing - i.e. a through-hole mount - or via tabs located around the sides of the battery holder.

Chassis mount battery holders can be either metal or plastic, open faced or enclosed (i.e. with or without a cover), and are commonly designed to accept a wide range of cell sizes including AA, AAA, C, D and 9V PP3

Panel Mount Battery Holders

A panel mount battery holder is designed to be slotted into a pre-existing cavity in the side, top or back exterior panel of a device, and removed entirely as a single self-contained unit when it`s time to swap out the batteries. They`re often manufactured such that the cover of the battery holder will sit flush with the device`s panel once inserted

Panel mount battery holders are most commonly secured in place via one of two methods - either via a screw-in system for optimal stability or with a flange for easier removal and reinsertion.


PCB Mount Battery Holders

As the name implies, a PCB mount battery holder is intended for applications where the battery holder needs to be installed directly onto a printed circuit board

The most common design is for the PCB mount holder to include several small sharp pins around its perimeter - these are pushed directly through the surface of the PCB and then soldered on from the rear side

Holders for PCB mounted batteries can be made from metal or plastic. While they`re frequently built to accommodate single coin cells, such as you might find on a typical computer motherboard, they can also be configured to allow for multiple cells and other battery types, including cylindrical AAA, AA, C, or D cells


Slide In Mount Battery Holders

As noted above, a slide-in mount battery holder will generally refer to the way the individual cells are inserted into and removed from the battery holder as a whole.

Slide-in mounts are arguably the most common type of battery connection/terminal setup, requiring the user simply to `drop` the cells into place either lengthways or side-on, at which point they`re either held in place by opposing forces from the two terminals, or secured with a cover or cap

Slide-in battery mounts tend to be among the cheapest and most cost-effective choices to buy in the UK for quickly and simply inserting a cell into an appropriate holder. They work reliably well, and are most often seen in molded plastic or metal battery holders incorporating self-contained battery compartments.


Snap-In Mount Battery Holders

A snap-in mount battery holder (sometimes called snap-on, depending on the manufacturer or supplier) denotes a terminal/contact setup where the cell is held more securely in place.

This is usually achieved using tensioned metal clips, sometimes incorporating springs, through which the battery needs to be pushed with a small amount of force before it clicks into place ready for use. Snap-in battery mounts give the device greater protection against accidental cell power loss during movement, vibration and shock

The term `snap-in` or `snap-on` is also widely used to refer to the standard contact types used on 9V cells (see Strap and lead battery holders, below).


Strap And Lead Battery Holders

Strap and lead battery holders aren`t actually holders per se, but rather a method of directly attaching a freestanding cell to an electronic device or motor without the use of an entire housing unit.

They generally consist of a wire, which is designed to be hooked up to the circuitry of the device via tinned ends, combined with a flat plastic terminal pad that attaches to the contacts of a battery via two press studs

The strap and lead battery holder type is most often used with 9V PP3 cells, allowing the cell to be lifted clear away from the main body of the item for removal, replacement or reattachment.

Surface Mount Battery Holders

A surface mount battery holder is designed to sit flat on any internal or external panel or plate, either around or inside the device being powered, and is generally open-faced.

Holders of this type are most often seen as a mounting solution for coin-type batteries with button terminal contacts, in applications where the cell doesn`t need to be connected directly to a PCB.

Types of battery holders
Once the correct size and mount style of battery holder has been chosen for your intended application and device, it`s also important to be aware of the different physical designs available for various battery holder types. In the section below, we`ll outline some of the more common versions.


Clip battery holders
A battery holder clip is a simple, economical choice for mounting batteries securely within a holder. They tend to consist of two metal arms, arranged in a loop with a small gap at the apex, through which the cells are pushed and subsequently held in place. Clip battery holders tend to be most often associated with common cylindrical batteries such as AA and AAA, but are also sold for securing C, D and 9V PP3 cells into position.


Contact battery holders
Contact battery holders refer to the simple tension-type terminal contact plates and accessories (including coil springs, button, top spring arm and leaf spring contact types) that hold a battery in place while allowing it to deliver power to the device drawing from it.

Again, these are cheap and cost-effective components widely available as separate swap-in or replacement parts that can be used to fit a very wide range of different cell sizes and battery holder types.


Through Hole Surface Mount Battery Holders

A through-hole surface mount battery holder is a common variation on the type outlined above, except it`s affixed to the surface of the device in question via a hole in the housing rather than via tabs around the perimeter. This makes the through-hole version better suited to applications where space is at a premium.


Wire Lead Mount Battery Holders

Wire lead mount battery holders denote a particular type of external connection between the battery holder itself and the circuitry of the device being powered. A wire lead mount battery holder will be supplied with two leads attached to the holder at one end, with the other ends free to be hooked into the circuitry of the device as the holder is attached.

This can offer greater flexibility in terms of battery holder mounting locations than certain pin or foot connection types, which must make immediate contact with the circuit wherever the holder is positioned.

Battery Holders

ATKCONN ELECTRONICS CO., LTD , https://www.atkconn.com