CodeWhisperer's features are becoming a part of Amazon Q Developer. Learn more
Evaluating and optimizing your customization
| The CodeWhisperer Customizations feature is in preview, and is subject to change. |
Evaluating your customization
This section explains how to evaluate your customization.
-
In the CodeWhisperer console, from the navigation panel, choose Customizations.
-
Choose the name of the customization to examine.
-
The right side of the window will display an evaluation score. This score indicates CodeWhisperer's evaluation of how effective your customization may be.
With your evaluation score in mind, you must now consider whether or not to activate your customization. In making this decision, take the following factors into consideration.
-
Very good 8-10: CodeWhisperer recommends that you activate this customization.
-
Fair 5-7: CodeWhisperer recommends that you activate this customization.
If you do not see significant improvement, consider the optimization suggestions below. If those are not effective, consider switching to a different code source.
-
Poor 1-4: This customization is not likely to be useful. Consider the optimization suggestions below. If those are not effective, consider switching to a different code source.
Optimizing your customization
This section contains suggestions for optimizing your suggestion in order to achieve a higher evaluation score.
-
Consider expanding your data source to include more code repositories.
-
If you primarily included data from limited programming languages, consider expanding to more languages.
-
Remove auto-generated files and repositories, or those generated from templates. Training a customization to generate or complete such files is typically not valuable, and tends to just add noise.
Note
CodeWhisperer automatically filters out non-code files, such as configuration files and text files.
-
It is possible that your codebase does not frequently use internal libraries. If you know this to be true, then the core CodeWhisperer model may already have been performing as well as possible.
Optimizing for the languages you use
In order for code in a particular language to be used in a customization, you must include at least 10 data files containing that language, and all of your source files together must come to at least 20 MB. If your developers write code in a language that is not supported by your customization, CodeWhisperer's recommendations in that language will come from the CodeWhisperer base model (not your customization). In other words, they will be the same recommendations that you would receive if you did not have a customization. This, in turn, could affect the metrics on your dashboard. For example, the "Lines of code generated by CodeWhisperer" may be less than what it would have been if the language commonly used by your developers had been included in your customization.
