Serendipity35

Last year, MIT researchers announced that they had built “liquid” neural networks, inspired by the brains of small species. This is a class of flexible, robust machine-learning models that learn on the job and can adapt to changing conditions. That is important for safety-critical tasks, like driving and flying.

The flexibility of these “liquid” neural nets are great but they are computationally expensive as their number of neurons and synapses increase and require inefficient computer programs to solve their underlying, complicated math.

Now, the same team of scientists has discovered a way to alleviate this bottleneck by solving the differential equation behind the interaction of two neurons through synapses. This unlocks a new type of fast and efficient artificial intelligence algorithm and is orders of magnitude faster, and scalable.

What I find interesting is that the equation that needed to be solved to do this has not had a known solution since 1907. That was the year that the differential equation of the neuron model was introduced. I recall when I was a student and when I was teaching at a university (in the humanities) hearing the complaints of students battling away in a course on differential equations.

These models are ideal for use in time-sensitive tasks like pacemaker monitoring, weather forecasting, investment forecasting, or autonomous vehicle navigation. On a medical prediction task, for example, the new models were 220 times faster on a sampling of 8,000 patients. 

Specifically, the team solved, “the differential equation behind the interaction of two neurons through synapses… to unlock a new type of fast and efficient artificial intelligence algorithms.” “The new machine learning models we call ‘CfC’s’ [closed-form Continuous-time] replace the differential equation defining the computation of the neuron with a closed form approximation, preserving the beautiful properties of liquid networks without the need for numerical integration,” MIT professor and CSAIL Director Daniela Rus said. By solving this equation at the neuron-level, the team is hopeful that they’ll be able to construct models of the human brain that measure in the millions of neural connections, something not possible today. The team also notes that this CfC model might be able to take the visual training it learned in one environment and apply it to a wholly new situation without additional work, what’s known as out-of-distribution generalization. That’s not something current-gen models can really do and would prove to be a significant step toward the generalized AI systems of tomorrow.

Source  https://news.mit.edu/2022/solving-brain-dynamics-gives-rise-flexible-machine-learning-models-1115

Albert Einstein was definitely a subject matter expert, but he is not regarded as a good professor. Einstein first taught at the University of Bern but did not attract students, and when he pursued a position at the Swiss Federal Institute of Technology in Zurich, the president raised concerns about his lackluster teaching skills. Biographer Walter Isaacson summarized, “Einstein was never an inspired teacher, and his lectures tended to be regarded as disorganized.” It's a bit unfair to say that "Einstein Was Not Qualified To Teach High-School Physics" - though by today's standards he would not be considered qualified. It probably is fair to say that "Although it’s often said that those who can’t do teach, the reality is that the best doers are often the worst teachers."

Beth McMurtrie wrote a piece in The Chronicle called "What Would Bring the Science of Learning Into the Classroom?" and her overall question was: Why doesn't the scholarship on teaching have as much impact as it could have in higher education classroom practices?

It is not the first article to show and question why higher education appears not to value teaching as much as it could or should. Is it that quality instruction isn't valued as much in higher education as it is in the lower grades? Other articles show that colleges and most faculty believe the quality of instruction is a reason why students select a school.

Having moved from several decades in K-12 teaching to higher education, I noticed a number of things related to this topic. First of all, K-12 teachers were likely to have had at least a minor as undergraduates in education and would have taken courses in pedagogy. For licensing in all states, there are requirements to do "practice" or "student teaching" with monitoring and guidance from education professors and cooperating teachers in the schools.

When I moved from K-12 to higher education at NJIT in 2001, I was told that one reason I was hired to head the instructional technology department was that I had a background in pedagogy and had been running professional development workshops for teachers. It was seen as a gap in the university's offerings. The Chronicle article also points to "professional development focused on becoming a better teacher, from graduate school onward, is rarely built into the job."

As I developed a series of workshops for faculty on using technology, I also developed workshops on better teaching methods. I remember being surprised (but shouldn't have been) that professors had never heard of things like Bloom's taxonomy, alternative assessment, and most of the learning science that had been common for the past 30 years.

K-12 teachers generally have required professional development. In higher education, professional development is generally voluntary. I quickly discovered that enticements were necessary to bring in many faculty. We offered free software, hardware, prize drawings and, of course, breakfasts, lunches and lots of coffee. Professional development in higher ed is not likely to count for much when it comes to promotion and tenure track. Research and grants far outweigh teaching, particularly at a science university like NJIT.

But we did eventually fill our workshops. We had a lot of repeat customers. There was no way we could handle the approximately 600 full-time faculty and the almost 300 adjunct instructors, so we tried to bring in "champions" from different colleges and departments who might later get colleagues to attend.

I recall more than one professor who told me that they basically "try to do the thing my best professors did and avoid doing what the bad ones did." It was rare to meet faculty outside of an education department who did any research on teaching. We did find some. We brought in faculty from other schools who were researching things like methods in engineering education. I spent a lot of time creating online courses and improving online instruction since NJIT was an early leader in that area and had been doing "distance education" pre-Internet.

Discipline-based pedagogy was definitely an issue we explored, even offering specialized workshops for departments and programs. Teaching the humanities and teaching the humanities in a STEM-focused university is different. Teaching chemistry online is not the same as teaching a management course online.

Some of the best parts of the workshops were the conversations amongst the heterogeneous faculty groups. We created less formal sessions with names that gathered professors around a topic like grading, plagiarism and academic integrity, applying for grants, writing in the disciplines, and even topics like admissions and recruiting. These were sessions where I and my department often stepped back and instead offered resources to go further after the session ended.

It is not that K-12 educators have mastered teaching, but they are better prepared for the classroom from the perspective of discipline, psychology, pedagogy, and the numbers of students and hours they spend in face-to-face teaching. College faculty are reasonably expected to be subject matter experts and at a higher level of expertise than K-12 teachers who are expected to be excellent teachers. This doesn't mean that K-12 teachers aren't subject matter experts or that professors can't be excellent teachers. But the preparations for teaching in higher and the recognition for teaching excellence aren't balanced in the two worlds.

I learned about Gordon Moore's 1965 prediction about 10 years after he proposed it. He said that by paying attention to an emerging trend, he extrapolated that computing would dramatically increase in power, and decrease in relative cost, at an exponential pace. His idea is known as Moore’s Law. Moore's law sort of flips Murphy's law by saying that everything gets better.

Moore was an Intel co-founder and his idea was "law" in the electronics industry. Moore helped Intel to make the ever faster, smaller, more affordable transistors that are in a lot more than just computers today. The 2021 chip shortage globally reminded us that cars and appliances and toys and lots of other electronics rely on microchips.

Moore's law is the observation that the number of transistors in a dense integrated circuit (IC) doubles about every two years. (Originally, Moore said it would happen every year but he revised it in 1975 when I was introduced to it to say that it would happen every two years.)

Though the cost of computer power for consumers falls, the cost for chip producers rises. The R&D, manufacturing, and testing costs keep increasing with each new generation of chips. And so, Moore's second law (also called Rock's law) was formulated saying that the capital cost of a semiconductor fabrication also increases exponentially over time. This extrapolation says that the cost of a semiconductor chip fabrication plant doubles every four years.

Huang's Law is new to me. Up front, I will say that this newer "law" is not without questions about its validity. It is based on the observation that advancements in graphics processing units (GPU) are growing at a rate much faster than with traditional central processing units (CPU).

This set Huang's Law as being in contrast to Moore's law. Huang's law states that the performance of GPUs will more than double every two years. The observation was made by Jensen Huang, CEO of Nvidia, in 2018. His observation set up a kind of Moore versus Huang.  He based it on Nvidia’s own GPUs which he said were "25 times faster than five years ago." Moore's law would have expected only a ten-fold increase.

Huang saw synergy between the "entire stack" of hardware, software and artificial intelligence and not just chips as making his new law possible.

If you are not in the business of producing hardware and software, how do these "laws" affect you as an educator or consumer? They highlight the rapid change in information processing technologies. The positive growth in chip complexity and reduction in manufacturing costs would mean that technological advances can occur. Those advances are then factors in economic, organizational, and social change.

When I started teaching computers were not in classrooms. They were only in labs. The teachers who used them were usually math teachers. It took several years for other disciplines to use them and that led to teachers wanting a computer in their classroom. Add 20 years to that and the idea of students having their own computer (first in higher ed and about a decade later in K-12) became a reasonable expectation. During the past two years of pandemic-driven virtual learning, the 1:1 ratio of student:computer became much closer to being ubiquitous.

Further Reading
investopedia.com/terms/m/mooreslaw.asp
synopsys.com/glossary/what-is-moores-law.html
intel.com/content/www/us/en/silicon-innovations/moores-law-technology.html

Facebook has been getting a lot of critical press the past month.  The Wall Street Journal's "Facebook Files" series has focused attention on how Facebook Inc. knows from internal research that its three platforms allow content that causes harm and any actions it has taken have not been effective.

When they announced this summer that there is a project to develop a version of Instagram aimed at children younger than 13, there was an outcry in the media. Concerns about privacy, screen time, mental health and safety were all aired.

This week Facebook announced it is suspending plans to build the Instagram Kids app. Facebook has owned Instagram since 2012. The platform is largely a photo-sharing application, though it has the commenting and likes common to most social sites. The Wall Street Journal series covered how Instagram is known by Facebook to sometimes negatively affect teenage girls in particular.

This suspension is not an end to the project and the company plans to take some time to work with parents, experts, policymakers and regulators, but to move forward. Introducing the next generation to the platform would be advantageous to the company, though they had said that the Kids app would be ad-free, introducing kids to what may become in their adult life the Facebook "metaverse."

Facebook/Instagram/WhatsApp is certainly not alone in wanting new and younger users and is competing with other platforms such as TikTok and Snapchat.

It may seem somewhat ironic that the WSJ used the results of an internal study by Facebook which they conducted to determine how its apps affect users against the company. In fact, the WSJ did compliment Facebook on doing the research, but their criticism came in what Facebook did or did not do as a result of the studies.

Facebook is scheduled to address these issues this Thursday before the Senate Subcommittee on Consumer Protection, Product Safety and Data Security.

https://www.washingtonpost.com/business/2021/09/27/facebook-instagram-kids/
https://www.wsj.com/articles/facebook-pauses-instagram-kids-project-11632744879
https://www.engadget.com/facebook-is-pausing-work-on-instagram-kids-app-124639135.html

I took one course in statistics. I didn't enjoy it, though the ideas in it could have been interesting, the presentation of them was not.

I came across a video by Cassie Kozyrkov that asks "What if I told you I can show you the difference between Bayesian and Frequentist statistics with one single coin toss?" Cassie is a data scientist and statistician. She founded the field of Decision Intelligence at Google, where she serves as Chief Decision Scientist. She has another one of those jobs that didn't exist in my time of making career decisions.

Most of probably had some math teacher use a coin toss to illustrate simple probability. I'm going to toss this quarter. What are the odd that it is heads-up? 50/50. The simple lesson is that even if it has come up tails 6 times in a row the odds for toss 7 is still 50/50.

But after she tosses it and covers it, she asks what is the probability that the coin in my palm is up heads now? She says that the answer you give in that moment is a strong hint about whether you’re inclined towards Bayesian or Frequentist thinking.

The Frequentist: “There’s no probability about it. I may not know the answer, but that doesn’t change the fact that if the coin is heads-up, the probability is 100%, and if the coin is tails-up, the probability is 0%.”

The Bayesian: “For me, the probability is 50% and for you, it’s whatever it is for you.”

Cassie's video about this goes much deeper - too deep for my current interests. However, I am intrigued by the idea that if the parameter may not be a random variable (Frequentist) you can consider your ability to get the right answer, but if you let the parameter be a random variable (Bayesian), there's no longer any notion of right and wrong. She says, "If there’s no such thing as a fixed right answer, there’s no such thing as getting it wrong."

I'll let that hang in the air here for you to consider.

---

If you do have an interest to go deeper, try:
Frequentist vs Bayesian fight - your questions answered
An 8 minute statistics intro
Statistical Thinking playlist
Controversy about p-values (p as in probabllity)

 

I was having a Facebook conversation with a friend about how photos and videos change our memories. Kids who grew up in the past 30 years - and more so in the age of smartphones and social media - have definitely had their memories sculpted by images of their past. My sons have said to me several times when I ask them "Do you remember us being there?" that "I remember the photos of it." Do the photos trigger a memory to return or is the photo the memory itself?

I am fascinated by how memory works. Research shows that when we describe our memories differently to different audiences it isn't only the message that changes, but sometimes it's also the memory itself. Every time you remember an event from the past, your brain networks change in ways that can alter the later recall of the event. The next time you remember it, you might recall not the original event but what you remembered the previous time. This leads some to say that memory is like the "telephone game."

This sent me back to an article I read in 2017. I did a search and found it again since my memory of this article on memory may not be remembered correctly. It is titled "Facebook is Re-Sculpting Our Memory" by Olivia Goldhill. Facebook is not the only social network or the only place that we share photos and videos, but it is a major place for this sharing.

I have a new granddaughter and her parents have set up a shared photo album online for relatives. They don't want people (mostly me - the oversharer) to post photos of her on Facebook, Instagram et al. I understand that privacy caution. My granddaughter will have many thousands of photos and videos to look at one day. I have about two dozen black and white photos of my first two years of life. It is probably two 12 photo rolls of film from that time (the 1950s) which seemed like enough to my parents to chronicle my early life.

Those photos of baby me don't trigger any memories but they are my "memory" of that time along with my mother's narration. "That was your stuffed lamb that was your favorite toy."

I have also kept journals since my teen years. The way to chronicle life once was to write it down. Rereading those journals now is a mixed experience. For some things, the journal is now my memory. Without the entry, I couldn't recall names, places or details from 40 years ago. But for some entries, I know that the version I wrote at age 15 is a kind of augmented reality. I made some things sound better or worse than the actual event. I sculpted the memory. Maybe as my memory degrades, those entries - accurate or not - will become the only memory I have.

Those sculpted memories are not unlike the image of ourselves we put online. Not all, but many people, post almost exclusively the best parts of their lives. Alfred Hitchcock said "Drama is life with the dull bits cut out," and that's true of many virtual lives as portrayed online.

That article references Daniel Schacter, a psychology professor at Harvard University, whose 1990s research first established the effects of photographs on memories. Frighteningly, he showed that it was possible to implant false memories by showing subjects photos of an event that they might have experienced but that they didn’t experience.

Another of his experiments found that while looking at photos triggered and enhanced the memory of that particular event, it also impaired memories of events that happened at the same time and were not featured in the photographs.

This sounds terrible, but one positive effect he has found that comes from weaknesses in our memory helps allow us to think meaningfully about the future.

In our recent discussions about fake news and images and videos that are not accurate, we realize that these weaknesses in memory and the ability to implant memories can be very powerful and also very harmful. "Source information” is a weakness of memory that can be tapped for devious purposes. How often have you heard someone explain that they heard it or read it or saw it "somewhere?"  We commonly have trouble remembering just where we obtained a particular piece of information. Though true off-line, for online information we may recall a "fact" but not the source - and that source may Online, this means we could easily misremember a news story from a dubious source as being from a more credible publication.

One phenomenon of memory is now called “retrieval-induced forgetting” I spent four years living at my college but I have a limited number of photographs from the time. Those photos and ones in yearbooks and some saved campus newspapers, plus my journal entries are primarily what I recall about college life. Related things that I can't review are much harder, if not impossible, to remember.

Social media is certainly sculpting (or perhaps resculpting) our memories. Is this making our ability to remember worse? That's not fully determined as of now. Nicholas Carr wrote a book called The Shallows: What the Internet Is Doing to Our Brains that looked at some neurological science in an attempt to see the impact of computers and the Net and that is certainly related to but not exactly the same as memory and images. The controversial part of Carr's book is the idea that the Internet literally and physically rewires our brain making it more computer-like and better at consuming data. But a surprisingly large section of the book is devoted to the history of the written word and all that it has done to “mold the human mind.”

Facebook, Instagram, TimeHop and other tools are reminding me daily of memories from years past. At times, I think "Oh yes, we were in Prague on this day two years ago." Other times, I say to myself, "I don't remember writing this 4 years ago." I react the same way to my old journals and black and white photos in an album taken a half-century ago.