Memristors and Artificial Intelligence

From March 2023 IEEE Spectrum: https://spectrum.ieee.org/memristor-devices-ai

• "Memristors, or memory resistors, are essentially switches that can remember which electric state they were toggled to after their power is turned off."
• "They can be used to both compute and store data"
• This allows for a better neuron emulation and in tests yielded excellent power consumption per computation of multiply operations.
• Understanding how to simulate this type of circuit before fabrication is likely a hot future topic.
• What is impressive is the new design and manufacturing of these circuits.  Prior efforts had poor yields and reliability but the new ones have had yields of 100%.  This is from a team in Israel.
• The devices could withstand 100,000 cycles of programming and could retain data for more than 10 years.
• I wonder if the limitation is the 100,000 cycles of programming and erasing.  Isn't every signal that goes through programming and erasing? If a clock cycle is 1 ns and we assume that each memristor was utilized 1/10th of the time, then the entire circuit would only last 1 second! (1ns * 100,000 * 1/0.1).  I wonder how they will address this deficiency.
• Regardless this is way cool
• The next researches from France use memristors for Bayesian reasoning. They also used knowledge-based neural nets (KBNN) to solve recognition of handwriting tasks.
• "Bayesian reasoning is often thought of as computationally expensive with conventional electronics"  -- but the new work was able to perform the handwriting recognition with 1/800 to 1/5000 the energy.

Identification of hammerstein–wiener models

 Wills, A., Schön, T.B., Ljung, L. and Ninness, B., 2013. Identification of hammerstein–wiener models. Automatica, 49(1), pp.70-81.

https://www.diva-portal.org/smash/get/diva2:608221/FULLTEXT01.pdf

• They state that in the absence of v_t, that "the model is essentially an output-error one for which standard estimation methods are well established." While we don't have injected noise here, v_t represents both measurement and modeling errors.  For my application, it would be wholly modeling errors, in L, so this approach could be well suited to my problem domain.
• The modeling technique I'm using is to first model L utilizing small-signal (linear) signals and then use full range signals to model f_w.  What is interesting here is to the possibility to model all parameters simultaneously (and hopefully weight them as well).
• The paper gets heavy quick.  
• Maximum likelihood (ML) approach
• sequential importance sampling
• expectation-maximisation (EM) --> refer us to a book on the topic for background
• "incomplete data X"
• particle smoothers vs. particle filters (I don't know what either one are)
• At the very least, this paper has an extensive biography from which to draw from.

Fitting Gaussian

 K. Wu, J. A. Zhang and Y. J. Guo, "Fast and Accurate Linear Fitting for an Incompletely Sampled Gaussian Function With a Long Tail [Tips & Tricks]," in IEEE Signal Processing Magazine, vol. 39, no. 6, pp. 76-84, Nov. 2022, doi: 10.1109/MSP.2022.3194692.

• "Since the Gaussian function is underlain by an exponential function, it is nonlinear and not easy to be fitted directly. One effective way of counteracting its exponential nature is to apply the natural logarithm, which has been applied in transferring the Gaussian fitting into a linear fitting [4]. However, the problem of the logarithmic transformation is that it makes the noise power vary over data samples, which can result in biased Gaussian fitting. The weighted least square (WLS) fitting is known to be effective in handling uneven noise backgrounds [7]."
• This is great because I commonly use the natural logarithm for these but haven't been  hyper aware of the downsides.
• Iterative weighted least square (WLS) fitting
• difficulty is the convergence time of the algorithm
• Can be addressed by a good initialization of the parameter estimations
• "Our work is mainly aimed at designing a way of initializing the weight vector of the iterative WLS so as to reduce the number of overall iterations and relieve the dependence of fitting performance on the proportion of the tail region."

CTLE with Ferroelectric Capacitor

 D. Borah, A. Gupta, T. S. Kalkur and K. Miller, "A 5-Gb/s Adaptive Continuous Time Linear Equalizer Using Ferroelectric Capacitor," in IEEE Transactions on Components, Packaging and Manufacturing Technology, vol. 12, no. 4, pp. 647-654, April 2022, doi: 10.1109/TCPMT.2022.3158933.

• The introduction has a nice survey of the different techniques to adaptively select CTLE settings. "slicer-based power sensing method" and the "spectrum balancing method".  
• CTLEs typically use capacitors with discrete settings but they propose a barium strontium titanate (BST) capacitors that varies continuously. 
• The capacitor they consider "offers high linearity, high power capability, and low leakage current which makes it suitable for mobile phone applications and dedicated RF tunable applications."
• I stopped reading after the introduction.  This looks like a cool technology to keep an eye on.

Fast JTOL

 A. Viveros-Wacher, R. Baca-Baylón, F. E. Rangel-Patiño, J. L. Silva-Cortés, E. A. Vega-Ochoa and J. E. Rayas-Sánchez, "Fast Jitter Tolerance Testing for High-Speed Serial Links in Post-Silicon Validation," in IEEE Transactions on Electromagnetic Compatibility, vol. 64, no. 2, pp. 516-523, April 2022, doi: 10.1109/TEMC.2021.3122348.

Jitter Tolerance (JTOL) tests take a long time to run in the lab as evidenced by this table from the paper:

• ME: JTOL testing is a Bit error rate (BER) test while sweeping sinusoidal jitter amplitude and frequency.
•  XAUI, PCIe, USB and SATA all have slightly different JTOL tests
• "In this article, we present a novel approach to dramatically accelerate JTOL testing based on a modified golden section direct search optimization algorithm."
• "To perform a JTOL test, each protocol specification defines a calibration procedure prior to the JTOL execution. This procedure defines the specific random jitter (JR), bounded uncorrelated jitter (JBU), duty-cycle distortion, and ISI values injected to the test pattern, which remain constant throughout the JTOL test, while JP (jitter periodic) is varied in both amplitude and frequency."
• These guys use JTOL to validate the system, in SerDes Toolbox we would want to use JTOL to characterize the CDR during design.  But regardless, both are desired.
• "In the traditional way to run JTOL testing, at each frequency point, the value of JP is initialized at a sufficiently low starting point to guarantee a PASS result from the BERT. Then, JP is increased by a certain amount, typically equivalent to the minimum value allowed by the BERT equipment for best accuracy. Then, a test is performed at the compliance BER. This is iteratively done until the BERT yields a FAIL. The result reported at each frequency point is the last JP value that yields a PASS.
• For a given frequency their algorithm adjusts SJ amplitude until the desired BER is achieved.  
• Overall an interesting idea to consider.

HilbertNet: A Probabilistic Machine Learning Framework for Frequency Response Extrapolation

 O. W. Bhatti, H. M. Torun and M. Swaminathan, "HilbertNet: A Probabilistic Machine Learning Framework for Frequency Response Extrapolation of Electromagnetic Structures," in IEEE Transactions on Electromagnetic Compatibility, vol. 64, no. 2, pp. 405-417, April 2022, doi: 10.1109/TEMC.2021.3119277.

• Wow! this looks like a useful NN for signal integrity applications.
• Introduction has a very through survey of S-parameter extrapolation techniques.  
• "Knowledge-based neural networks are used in [11] to provide extrapolated results for the design space parameters like physical length and width of copper traces. While this approach works for design space, it may not be applicable to extrapolation."
• RNN: Recurrent Neural Nets
• Predict the next sample given the previous batches of samples.
• I like this emphasis on providing uncertainty bounds:  
• "Historically, neural networks have been used to provide point estimates of their predictions. Such models lack explainability and could generalize incorrectly in the extrapolated space. Hence, it becomes necessary to provide uncertainty estimates around our predictions. This not only provides an insight to the model accuracy outside the training data but also enables the designer to choose whether to simulate more training points.
• This is a nice summary 
• "Our contributions include the following. 
• 1) Use of specialized long short term memory recurrent neural networks (LSTM-RNN) for frequency response extrapolation
• 2) HT (Hilbert Transform) to correlate real and imaginary part of the signal to enable causal complex-valued extrapolation.
• 3) Harnessing the variational inference-based Bayesian approach to assess the uncertainty of predictions in the extrapolated space.

Section II: Hilbert Transform

• They provide a nice algorithm to compute the discrete Hilbert Transform

Section IV

• Break the response up into shorter sliding sequences and train the RNN on using earlier sequences to predict later sequences.
• I like this because you are training the NN on the data to be extrapolated, not on a boat load of data that could or not be relevant to the particular problem at hand.
• This is the training phase
• Training -> Inference phase

Section IV:B Bayesian Recurrent Network Architecture

• Great explanations.  After two read throughs I think I have a good idea of what they are doing.  I'd really like to learn more about probabilistic estimation/detection theory
• Is able to extract the variance of the estimation so the uncertainty bound is tracked.

Above is their result from applying this to a PDN network.  Not only were they able to capture the resonances but they were able to provide a confidence bound to the prediction! Brilliant.  They also provide a link to some Python code.

Overall, this is very impressive.  It uses the dataset itself as all of the training data and yields a very compelling result. Their application is to predict high frequency behavior and then let the user decide if they need to simulate to these higher frequencies or not.  

My high frequency extrapolation is so I can convert to the time domain for a time response of a given time step.  Within my use case I wonder if the insights provided by this paper can be of use to me.

The Sirens of Mars: Searching for Life on Another World.

 Johnson, Sarah Stewart. The Sirens of Mars: Searching for Life on Another World. Crown, 2020.

Great popular science book about the human journey to discover what Mars is like.  It is also a personal memoir of the scientist Sarah Stewart Johnson and one that I would recommend to any young woman regardless if they have ambitions for science careers or not.