Finally, a neural network has emerged, which is really important to be understood.
The performance of the processor often rests not on computing blocks, but in memory. The system loses the time when there is no necessary data in the cache and the processor has to go further along the memory chain. Researchers from the University of North Carolina have proposed a CacheMind tool that helps disassemble such failures not by summary figures, but for reasons. The system analyzes the behavior of the cache, answers questions in natural language and tells you exactly where the memory architecture loses speed.
The cache stores the data that the program will probably request again soon. The meaning is simple: take data from the cache faster than read them from other levels of memory, and even more so from the drive. The problem is the amount. The cache can not keep everything at once, so the architects constantly decide which data should be uploaded in advance, and what time to displace.
Two mechanisms are used to speed up the work. The first is called pre-sampling. The system pulls up in advance into the cache data that may soon be needed. The second mechanism is called a substitution policy. The algorithm decides which block needs to be removed to make room for a new one. The error in any part of this scheme hits performance: the program is more likely to miss the cache, is waiting longer for the data and works more slowly.
The authors explain their task as follows: it is difficult to optimize the substitution policy, because an engineer needs to understand what blocks of data will need in the near future. There is little to know the general statistics. Need details on the level of individual instructions and memory appeals: what command depends on the data that is not in the cache, what appeals cause the displacement chain, which areas of the program interfere with each other.
The usual process of working with simulators is as follows: the architect starts the model, receives aggregated indicators, changes the selection or substitution parameters, then re-starts the calculation and looks, has become better or worse. The approach with the search of options shows the result, but almost does not explain the cause. The engineer sees the numbers, but does not see the internal logic of the mistakes.
CacheMind works differently. The system uses a cause-and-effect analysis and helps not just fix the problem, but to disassemble where it came from. Instead of another round of trial and error, the architect may ask why the reversal of memory associated with a specific team counter causes more displacement. The tool should answer not with a general phrase, but a full-fledged detailed report.
The track in this context is a detailed summary of how the program turned to memory during work. The record can be traced what instructions followed each other, what data fell into the cache, which blocks were supplanted and at what point the productivity loss began.
The developers separately emphasize that the main goal was not only to analyze the internal operation of the processor, but also to explain the reasons. The project was conceived as a convenient tool for an architect who needed not just a simulator report, but an opportunity to ask an arbitrary question and get a meaningful answer. This setting has complicated the development: ordinary language models learn to answer in pre-known examples of questions and answers, and here you need to maintain a free dialogue on a complex engineering topic.
In the tests for proof of the concept, CacheMind improved all the test cases in two indicators at once. The authors recorded an increase in the share of hits in the cache and acceleration of the implementation of programs. Precision values in official sources are not given, but the researchers claim that the positive effect manifested in all tests.
Since CacheMind is the first tool based on a large language model, which is specially designed to work with cache replacement politicians, the team prepared a separate benchmark CacheMindBen. The set consists of 100 questions about substitution policies with verified responses. It is needed to compare future systems that will solve a similar problem.
Verified sets of tasks are also important because they serve as examples for contextual learning. In machine learning, this technique is called a few-shot learning: the model receives several samples and learns to answer new, unprecedented questions, based on the context.
CacheMindBench gives language models a context that helps them reproduce reasoning in an object field in which the model was not taught in advance. Due to this, CacheMind can be used on the principle of plug and play: connect to a new configuration, a new question or a new software load without additional training.
Moreover, the authors directly write that the capabilities of CacheMind and CacheMindBench go beyond the substitution policies and can be useful for a wider range of tasks of computer architecture, where the engineer needs not only the result of the simulation, but also the explanation of why the system behaves this way.
The performance of the processor often rests not on computing blocks, but in memory. The system loses the time when there is no necessary data in the cache and the processor has to go further along the memory chain. Researchers from the University of North Carolina have proposed a CacheMind tool that helps disassemble such failures not by summary figures, but for reasons. The system analyzes the behavior of the cache, answers questions in natural language and tells you exactly where the memory architecture loses speed.
The cache stores the data that the program will probably request again soon. The meaning is simple: take data from the cache faster than read them from other levels of memory, and even more so from the drive. The problem is the amount. The cache can not keep everything at once, so the architects constantly decide which data should be uploaded in advance, and what time to displace.
Two mechanisms are used to speed up the work. The first is called pre-sampling. The system pulls up in advance into the cache data that may soon be needed. The second mechanism is called a substitution policy. The algorithm decides which block needs to be removed to make room for a new one. The error in any part of this scheme hits performance: the program is more likely to miss the cache, is waiting longer for the data and works more slowly.
The authors explain their task as follows: it is difficult to optimize the substitution policy, because an engineer needs to understand what blocks of data will need in the near future. There is little to know the general statistics. Need details on the level of individual instructions and memory appeals: what command depends on the data that is not in the cache, what appeals cause the displacement chain, which areas of the program interfere with each other.
The usual process of working with simulators is as follows: the architect starts the model, receives aggregated indicators, changes the selection or substitution parameters, then re-starts the calculation and looks, has become better or worse. The approach with the search of options shows the result, but almost does not explain the cause. The engineer sees the numbers, but does not see the internal logic of the mistakes.
CacheMind works differently. The system uses a cause-and-effect analysis and helps not just fix the problem, but to disassemble where it came from. Instead of another round of trial and error, the architect may ask why the reversal of memory associated with a specific team counter causes more displacement. The tool should answer not with a general phrase, but a full-fledged detailed report.
The track in this context is a detailed summary of how the program turned to memory during work. The record can be traced what instructions followed each other, what data fell into the cache, which blocks were supplanted and at what point the productivity loss began.
The developers separately emphasize that the main goal was not only to analyze the internal operation of the processor, but also to explain the reasons. The project was conceived as a convenient tool for an architect who needed not just a simulator report, but an opportunity to ask an arbitrary question and get a meaningful answer. This setting has complicated the development: ordinary language models learn to answer in pre-known examples of questions and answers, and here you need to maintain a free dialogue on a complex engineering topic.
In the tests for proof of the concept, CacheMind improved all the test cases in two indicators at once. The authors recorded an increase in the share of hits in the cache and acceleration of the implementation of programs. Precision values in official sources are not given, but the researchers claim that the positive effect manifested in all tests.
Since CacheMind is the first tool based on a large language model, which is specially designed to work with cache replacement politicians, the team prepared a separate benchmark CacheMindBen. The set consists of 100 questions about substitution policies with verified responses. It is needed to compare future systems that will solve a similar problem.
Verified sets of tasks are also important because they serve as examples for contextual learning. In machine learning, this technique is called a few-shot learning: the model receives several samples and learns to answer new, unprecedented questions, based on the context.
CacheMindBench gives language models a context that helps them reproduce reasoning in an object field in which the model was not taught in advance. Due to this, CacheMind can be used on the principle of plug and play: connect to a new configuration, a new question or a new software load without additional training.
Moreover, the authors directly write that the capabilities of CacheMind and CacheMindBench go beyond the substitution policies and can be useful for a wider range of tasks of computer architecture, where the engineer needs not only the result of the simulation, but also the explanation of why the system behaves this way.
