Sitemap
A list of all the posts and pages found on the site. For you robots out there, there is an XML version available for digesting as well.
Pages
Posts
Adventures in Vibe Coding a github.io website
Published:
I decided to create a github.io (https://lluang.github.io) website using academicpages.github.io framework, because that is what all the cool ML/AI kids seem to do these days. And it seemed that having a professional web presence there is better than just my blogger site (which is no longer cool. and has lots of non-professional musings). But it seems like a lot of work. A rendered site requires lots of data files. And websites are not my thing. But working with Generative AI is kinda my thing. So I was going to (1) use generative AI to customize my fork of the academicpages site, (2) create the data files for presentations, publications, and media appearances by pulling it out of the long form of my CV, and (3) vibe code the transfer of my blogger posts into markdown format so it would populate my github.io site.
Why learn theory
Published:
Claude’s Constitution on the role of the analyst
Published:
As I was reading Claude’s Constitution, I was reminded of discussions on what the role of an analyst advising decision makers should be. One of the core beliefs is that the principal (the decision maker we were advising) needed frank advice on the topic at hand so that they can make informed decisions. What makes this hard is a strong pressure to either do what the principal asks for or to say what the principal wants to hear. The Claude Constitution (and generations of analysts in the liberal democracies) reject this and characterize its role as genuinely helpful, courageous honesty, and a commitment to the organization’s long-term well being.
Book review: Little Brother by Cory Doctorow
Published:
This was written in 2008, in a U.S. where a Department of Homeland Security has turned on Americans in the name of security following a terrorist attack. And as its focus and energy was on keeping Americans in line, it created opposition in the form of people who wanted to go about their lives and do so without surveillance. And in this, the protagonists are teenager who were caught up in a dragnet while playing games.
What is needed to work in the age of Generative AI
Published:
Last week I was at CMU-Heinz for a fireside chat type event with students in the various MS in Analytics programs there. One question that I got was what were the skills needed to succeed in an environment with AI, and even into the future. Then I spoke about being able to program because you need to learn how to think deliberately, being able to connect technical capabilities with end business needs (because this has always been how analytics fails), and as I think about it more having a better understanding of knowledge. Because if you believe that your education and training is about learning sets of facts and recipes, AI will eat you alive. So your understanding of your field has to be greater than facts and procedures.
First, why learn computer programming when AI can write code faster than you. Microsoft has a set of studies showing a high (40%) rate of errors, yet their programmers also say they are more productive. Because, as I use AI at work, I find that it is helpful in creating good structure and framework scaffolding, especially when I have to context switch (I regularly switch between three data stacks at work, each of them have many people who spend all their time in one) or if I am applying methodologies new to me or my organization. But because I am competent, I can correct it as I go, and the fact that there were originally errors is not a big concern, because I was going to revise everything anyway
Another reason to learn programming is you learn to think in a different way. The ancient Greek philosophers had students learn geometry before philosophy. Not because geometry and math is beautiful (even though they are), but because with geometry comes proofs. And geometric proofs is about how much you can understand starting with a minimum amount of assumptions (Euclid’s five axioms). And you now have experience in determining an objective truth, no appeals to authority, no claims of different point of view. And your logic is in the open, to be critiqued on their own merits. Far different from my friend in grad school who claimed that perception is reality. And only then were you fit to move into the realm of ideas, where even facts have to be evaluated.
Programming languages differ from natural languages in their precision. Every statement has a single clear meaning. And this is different than natural languages, where the ambiguity of human life plays a role. So to work with anything regarding computers, it is helpful to recognize that computers will work with language in different ways we do, it handles ambiguity differently than people, and how it will use randomness to handle the difference (which is key to how Generative AI works).
The next is the link between the capabilities of technology and the needs of the business. According to everyone who has studied project failures in depth, failed communications between the business partner and the analysts is the biggest cause of project failure. And data projects have a failure rate between 80-90% (this range has been persistent in studies over decades in the data world, and it is consistent across definitions of failure and different segments in data analytics, data engineering, or data reporting (dashboards)). Being able to understand the business needs of end customers as well as understanding potential classes of technology solutions leads to asking better questions and getting value out of the applications of technology. The main reason for breakdowns in communication is ego and arrogance. From the technology side, there is often a belief that the customers are idiots who do not know what they want, so the technology people should just build something and pitch it back to the customers. This is mirrored by business people who think that technology is a a turnkey product so they should not interact with the people who are creating the solution. A third variation is when upper leadership decides to act as an intermediary between the analysts and the end customer. The logic here is generally that the leader believes both the technologists/analysts and the end customer have no communications skills, therefore the leader will handle all of the communications and give the requirements to the analysts. All of these are wrong. Especially in anything involving data, details matter, and the entire project involves discovery of details that no-one realized were important at the beginning. So the analyst and the end customer need to be regularly reviewing these discoveries, and adapting along the way. And only the end customer (because they are closest to the problem on the ground and know what kinds of actions can be taken) and the analyst (because they will be representing the detail in models and they know what the range of alternative models can do) together can make those decisions. Without that direct communication (potentially facilitated by someone who knows both sides), a project falls into the trap of solving the wrong problem. And this requires people who understand purpose and can determine the impact of nuance, both of which Generative AI does badly in.
The third category of future work is understanding your field. Computers are very good at retrieving facts, if those facts are in its knowledge base. Gen AI is better than prior technologies because it is not as sensitive to getting the wording precise. Computers are also very good at following instructions if those instructions are given. (people also tend to be better at things when they are given good instructions). So, if this is the extent of your subject expertise, you are in trouble. In software development, there is actually a very large workforce like this, whose careers are built on the ability to fill out a given framework or instructions. But if your place in the world is built on more than knowing facts or following recipes, if there is actual understanding that has to be applied on a situation specific basis, there is still room for you. Without that understanding, an organization can execute perfectly, a solution to the wrong problem. Which is worthless. So you need the level of understanding that allows for good judgement, and you need to be working in an organization that allows for its employees to use that judgement.
A last criteria is based on a number of conversations I’ve had. Many people express that they believe in the answers Generative AI gives because of the massive investment these companies have made, the smart people they have hired, and the belief that these companies would ensure correctness. I had to explain that these are industries and communities that have historically claimed they had no interest in accuracy or correctness. And until recently, viewed the paying customer as king and only sought to full the demand. Ethics was not part of the conversation. And what they delivered, did not come with guarantees other than it does what it does. This attitude that you did not question the authority that came with wealth and success is the first thing that has to be broken before people can use Gen AI productively. Both of my kids do it. I design the rollout and presentation of projects at work to make sure my business partners who are using Gen AI view its output skeptically and looking for specific types of flaws. As an avid reader of science fiction over the years, much of which addresses AI as part of society, and worry much less about the power of AI than I do about people who use the output of AI without being critical thinking. It is the kind of following that leads people to enact policies without analysis, and punishes people. And the outcomes are the fault of the people who followed the AI. Because AI has no goals, purpose, or conscience beyond that of its user.
Book review: Tools and Weapons: The Promise and the Peril of the Digital Age by Brad Smith
Published:
Tools and Weapons: The Promise and the Peril of the Digital Age by Brad Smith
My rating: 4 of 5 stars
The author, Brad Smith, was General Counsel for Microsoft (he was also President of Microsoft, but his role as counsel is more relevant for this book). The book is a discussion of privacy in a context where governments and large corporations hold immense amounts of personal and business data, and there is a large temptation for corporations to take advantage of that knowledge or governments to access that information, for governments in pursuit of legal action or suppression. So much of the book is about cases that involved Microsoft and how they developed a stance on corporate responsibilities to their customers on privacy matters, specifically in cases where government demanded customer data.
Each chapter revolves around a policy argument that played out in public forums, regulatory, legislative, and in the courts in the U.S. and Europe. And it is in a backdrop where technology companies used to believe that as technology companies they did not have an interest in policy. But as companies became less sellers of goods and more providers of services, in particular of data storage and cloud based communications services, they became targets of government and criminal action to access customer data without consent. In each chapter Smith introduces the context, then introduces a historical principal that predated cloud computing, and he makes the argument that the choices and policies used to govern oud based computing services should be the same that governed the same type of services in the pre-digital age.
The overall philosophy he gives is that Microsoft is a custodian of customer’s data, not the owner. And as custodian it will protect the customer’s property (data). And throughout the book he identifies allies (who have similar philosophies of protecting customer’s/citizen’s property and privacy) who only differ in details. And those that he as to be contentious with, because they are seeking to use and profit from individuals data or desire access for investigations. (and Microsoft in these cases wants a transparent process for doing this what protects their customers, who have the rights of citizens/residents)
Clearly, Smith is proud off his work, and believes that protecting the privacy of Microsoft customers, even in the face of government pressure, is the right thing to do (with a procedure for governments to prevent harm to other citizen’s rights, life, or property. But he does acknowledge allies, Google and several European governments come across very well here. So a reader has to be mindful that he does have rose colored glasses on Microsoft’s journey in this topic.
I appreciate the view of a non-technology person on these topics. As he is a lawyer, his perspective is to look at issues that seem very new because of the pace of technology change, and recognize that the issues have existed and debated before the digital age. As the infrastructure is owned by multi-national corporations, the relative power of industry and government is different. But the idea that industry desiring to protect the interests of their customers and government desiring the safety of its citizens should align is one worth engaging in.
Reflections on the Advent of OR: Using Generative AI in Analytics and Agile Operations Research
Published:
In December 2025 I participated in the Advent of OR (https://adventofor.com) which was a 24 day exercise that guided participants through an optimization project. And instead of just solving problems and creating models, the Advent of OR walked through an entire project life cycle, using the INFORMS Analytics Framework.
While I am not a student or early career who was the target audience, I took part, and I had three goals.
Use a new programming toolkit. I used VS Code with R and Quarto. I usually use R Studio and I wanted to try R on VS Code. And I think Quarto is the future replacing Jupyter Notebooks for Python and a natural evolution from R Markdown.
Practice in optimization. In the Operations Research world, I am NOT an optimization person. My thesis was applied probability (queuing) and my methods research has been in simulation (one stream in ranking & selection and another stream in Bayesian methods for input modeling)
Using Generative AI. I wanted to see how generative AI does in an operations research project. And I wanted to do it right in a setting where I can give it references to guide it. Note: I have found that Generative AI favors descriptive statistics, machine learning, and hypothesis based statistics over other forms of analytics, so it needs some guidance.
Toolkit
I had to set up VS Code with the R extensions, Quarto (and extension), ompr and the glpk with associated R ROI packages, and to make sure everything worked, I download the repository for OR_using_R by Tim Anderson. Then to render the book (meaning I made sure all the code ran) I had to install texlive with xetex and extra fonts. Generative AI (I had Gemini CLI installed) was very helpful in all of the system administration tasks since it could figure out what was needed every time there was an error message.
Data analysis and Optimization
Working with the data sets, it read in the data, (I had to give it some corrections along the way to help it recognize the data types. When the data files were read in, it recognized that the data sets did not correspond in granularity. In the R markdown file it created, in addition to generating the code that read in the data and created summaries, it also identified a number of questions and concerns about the data and created questions for the stakeholder. This was a good set of questions that corresponded to what others put forward.
It also did well with the optimization. Given an optimization textbook, I first asked the generative AI for a mathematical formulation based on the project description. Similarly, it created a process for determining what kind of problem this was and worked through that process to determine that this was a linear programming optimization problem.
Next was a LP formulation using OMPR. The first formulation was straight forward. I went and had the GenAI break out the formulation into its own R script to enforce a separation of concerns between the data handling, optimization model, and output processing.
I generally also ask for docstrings as I go, and the Gen AI did this for both the model as well as various handling functions. I generally read the docstrings to ensure they say what I expected them to say. When it did not, since the docstrings were written based on the code, I took it to mean that the code was not right (this exposed a mistake in the initial formulation of the LP in OMPR). Similarly, I had the Gen AI write unit tests for the constraints and a mock problem to test the optimization.
Agile Operations Research
One of the aspects of having the Advent of OR over 24 days is that it rotates topics between the art of modeling, implementing and managing models, and interactions with stakeholders. There are a couple of very important points. First is that interacting with stakeholders is not something that is at the beginning and end of project and ignored in the middle. There needs to be stakeholder engagement throughout the modeling process. A second point that has come out in the conversations on LinkedIn is that the most common cause of project failure across data analytics are communication failures, in particular between the analyst and the end customer. While this can have many causes (including management inserting themselves in between the analyst and end customer), as analysts we must have that direct interaction from the beginning of the project (business problem formulation in the INFORMS Analytics Framework)
In the early stages of the project, one factor we need to face is failure of imagination. The first level for analytics is that our stakeholders often do not know what is possible across the full range of analytics. Often a problem is presented as a request for a tool, but for the results of the project to have any value, it has to address the end problem, so business problem formulation has to start with the end problem, determine what kind of information from data can help the decision makers address the problem, and then we can start discussing what methods can provide results in the form that will be useful. Currently, because of media hype, the initial request can be for a dashboard, or a predictive model, or a generative AI tool. As operations research analysts we can also bring to bear statistics, forecasting, optimization, simulation, and queueing; and different ways of applying those methods to give different kinds of results that can be delivered to decision makers to make better decisions.
After the business problem formulation, the next big change in the project will occur when presenting the first minimum viable model to the end user. This is the first model that uses a minimum acceptable subset of the data and model that covers the most essential aspects of the smallest version of the problem. The reason this is important is before this, all conversations are abstract and theoretical. The first time a model with outputs is presented to an end user, the end user will start to imagine how they would use these results in real situations that have happened in the past. And they will start telling about all of the considerations they account for, the information they need to gather to make decisions, and who they need to consult and coordinate with. And this can change the entire project. And from experience, I do not think it matters how much work is done at higher levels to define the project, the first time a model is presented to an end user the project will change so that the outcomes can be usable to the business. So it is best to make that happen as early as possible so that change causes the least disruption to the work in progress.
The idea of rapid cycles of iteration and feedback from the customer, and the willingness to accept changes to the project due to that interaction are the hallmarks of agile development methodologies in the software development world. Having regular rounds of model iteration where additional elements are added to the model, and getting feedback from stakeholders to confirm that the project is on the right track to produce something useful. And just like the software development world has experienced, this is more likely to lead to useful product, and actually faster than attempting to follow a rigid path that leads to something irrelevant.
Conclusion
The Advent of OR proved to be a valuable exercise, offering a full-cycle project experience that highlighted two critical modern aspects of Operations Research: the integration of Generative AI and the necessity of an Agile approach. Generative AI demonstrated significant utility in accelerating system setup and basic modeling tasks, freeing up the analyst for higher-level problem-solving. More importantly, the experience reinforced that project success hinges on continuous, direct stakeholder engagement, mirroring the principles of Agile development. By prioritizing early delivery of a Minimum Viable Model, analysts can gain crucial feedback that aligns the project with real business needs, ultimately reducing the risk of communication-based failure and ensuring the final product is relevant and utilized.
Thoughts on mentoring within the analytics profession
Published:
Our careers and lives follow unique trajectories and structures. While we all have our own paths, it is helpful to have people who have gone ahead on similar paths to share experiences and thoughts on the future. Part of our professional development are mentorship relationships, which can be done in a wide range of settings, relationships, and time frames.
I am going to define mentoring as a longer term, unstructured professional relationship, with the focus of the relationship being the personal growth of the mentee. Typically, the basis of the relationship is that the mentor has gone on a path that the mentee is on themself, and the insights of time may be helpful for the mentee’s development.
One thing that distinguishes mentorship relationships from other professional relationships is that mentorship relationships are holistic. They look more than just the task at hand, or even a job position. The mentorship relationship may be career focused, but it will look at the whole person, and will recognize that overarching goals can change with life events, even life events outside their occupation. So, while a supervisor/manager can be a mentor, this is really not apparent until after the manager relationship has ended, and the relationship has become larger than the roles both individuals had when the relationship started.
As we all have unique life paths, we cannot expect that any one person has gone on the same path that we are on, but mentors bring not only their own life experience, but also the experiences of those whom they have lived life alongside. They have seen the decisions and choices of others, and how those decisions have advanced the goals, or not. They have seen people whose lives have taken them on different paths, and so have a broader view on what the future can hold than those whose view of the world is from the relatively structured life of home and school.
What topics come up? The focus on a mentorship relationship is on the growth of the mentee. In the context of technical professionals, this is the professional growth, but as part of a full life. So, with an understanding of the long term goals of the mentee, it can be working through broader issues on a project, such as other points of view. It can be soft skills or relational skills working with co-workers, superiors, juniors, or outside colleagues (customers, business partners, etc.). It can be suggestions on how to stretch as a person, to see and work through things from a broader perspective, and the skills needed to do this. A mentor can be a sounding board, providing different points of view (especially on the behalf of people who may not be good at communicate their point of view). It can be how to handle work/life balance, looking at a whole person. It could also include looking at alternative paths, that different positions or even career paths may be more suited for the goals of the mentee.
How does a mentorship relationship start? Like all relationships, you can never tell if a relationship is going to be long term at the beginning. But you have to begin somewhere. A first conversation is often about a particular topic, one that is of mutual interest. (and this initial meeting is sometimes arranged by organizations such as a company or a professional organization trying to promote mentorship among employees or members). After the first few conversations about that first topic, you should have observed if the relationship is broader than that first topic, and you can talk about if you want to continue meeting about topics as they come up.
What does the mentor get out of this relationship? Typically, people who are in mentoring relationships also have other rich relationships, which is how they get the background that makes them valuable as a mentor. Over time, the relationship becomes driven by both concern and curiosity about the other’s experiences in life. Often that includes issues that are more apparent to someone at an earlier stage of life or career. A mentorship relationship can then become one of an ongoing set of relationships that makes up a life well lived, and the ultimate hope, even when it is not an expectation, is that a relationship be one that lasts.
Do mentorship relationships last? Sometimes. Organizations such as workplaces and professional societies will often organize mentorship relationships, but these are always based on a topic of interest in the moment, and these relationships typically start out with short term boundaries. But, like all relationships, a short term relationship is what has potential to broaden into something longer. Does the relationship broaden beyond the topic where it was started? Do conversations evolve organically and feel natural when they branch into new topics? Over time, can the relationship feel like something that lasts as both sides grow and change (as all growing people do). So the transition from a formal, temporary relationship with a defined schedule and defined boundaries changes into something more long term and fluid. And a mentor/mentee relationships begins to feel more like professional colleagues, each moving through life and careers on adjacent paths.
Can mentorships relationships be informal? Yes, in the sense friendships are informal. In professional society meetings, it is common to see someone and immediately follow up from a conversation from a year ago, just like old friends. So you can have a relationship where you only see each other on occasion, but immediately pick up where you left off, just as old friends do. But the key is the long term relationship, that the conversations are about growing people, not only about topic at hand.
Are there aspects of Analytics that mentorship relationships are especially helpful? One area are the soft skills, the skills of working with colleagues, managers, and customers that is not part of the standard training of a technical professional. A mentor can relate to what the other person may be thinking and help the mentee develop that sense of empathy for others that make them more effective professionally. A second aspect is dealing with the hype that often accompanies the profession. The most recent example is the rise of Generative AI, but similar waves of publicity occurred around deep learning, big data, and machine learning in general. A mentor can place new ideas and concepts in the context of everything else a mentee knows, in contrast to teachers or thought leaders whose responsibility at any given point in time is single topic focused. A third aspect is a sense of what a mentee may need to be a well rounded professional. Training programs and classes tend to be singularly focused with a specific goal, but professional growth needs to be holistic, and designing such a path needs the attention of a person who is looking at the whole person.
Mentorship presents the potential of a valuable relationship, fostering personal and professional growth through a holistic and potentially long-term connection. It goes beyond task-oriented guidance, embracing the mentee’s whole person, from developing crucial soft skills and navigating career paths to contextualizing industry trends. While it can begin focused on specific topics or within formal programs, at their best mentorships evolve into enduring relationships, offering mutual benefits and enriching the lives of both mentor and mentee. In dynamic fields like Analytics, such relationships are particularly vital, providing the comprehensive support needed to cultivate well-rounded, effective professionals in changing times.
If you are interested in mentoring relationships, I would look to your professional society. If you are in analytics, I would recommend you look at INFORMS and their mentoring programs (Video on the value of mentoring in analytics) It is a professional society for advanced analytics (broadly defined) and is vendor, tool, and methodology neutral, which is important for a field that sees major changes over the course of decades.
Book review: AI Snake Oil by Arvind Narayanan and Sayash Kapoor
Published:
AI Snake Oil: What Artificial Intelligence Can Do, What It Can’t, and How to Tell the Difference by Arvind Narayanan
My rating: 4 of 5 stars
I read AI Snake Oil as part of the INFORMS Book Club. I work with predictive AI and generative AI at work, and I describe what I do as figuring out how AI fails, then work with my business partners to develop a process and application to make AI useful and productive. This book falls into the category of demonstrating how AI fails.
There are several chapters, each with a discussion of a way that AI fails and how the authors figured it out. But they have a pattern. First, the failures in AI is in part due to how a particular model is trained. If the training data does not match the intended use, such as the data actually represents one characteristic but the model is being used for something else. Next, they discuss that the people who made the model do not always have incentive to get it right. In particular, the large AI companies do not have incentive to either evaluate the quality of the models or improve them.
Some things I think they do well.
- Differentiate between various generations of AI. They specifically break out predictive AI, generative AI, and symbolic AI. Each of which work differently than the others.
- Focus on the training data. This is where AI models need to be examined (by definition, AI does not include a description of the system, so predictive and generative AI have to learn about the world through large amounts of diverse data.) And failures come from the data not matching the setting where a model is applied.
- Be skeptical of claims that come from computer companies. I always say don’t let people selling you things define terms. They also say don’t let industry set the rules, the standards, or barriers of entry. Because their goal is to defend their market share, not the benefit of society.
This is a good book to read, especially as part of a discussion. Highly recommended
Business problem framing: The value of frameworks in analysis and communiction
Published:
As part of ongoing professional education, I took the INFORMS Business Problem Framing class, which is also the lead in to the Certified Analytics Professional training that is being developed. You may wonder what can be learned in a 5 hour course. It is a tour of frameworks for looking at business problems. And this is something that gets short changed in engineering training that focuses on analytic methods (i.e.math) to the expense of understanding the problem in context and working with people. This course is needed by anyone who learned analytics as a math or computational practice and needs to learn to work with business partners/clients and is good for those who work with business partners/clients but want more range and ways of communicating (which is everyone who works in analytics in the real world)
Back when I was an enginneering professor, the supply chain center at the associated business school asked if I would be willing to coach their supply chain case study teams. As a school with a certain amount of pretension, they found it disheartening that they were never winning these regional competitions (operations management supply chain professors and industrial engineering supply chain professors are drawn from the same pool, so this actually made sense as they needed someone who was not actually part of running a case competition.) But, after some conversations I did not approach it as a refresher on supply chain modeling, I approached our coaching sessions as teaching them how to read a case through the lens of frameworks that they had learned at some point, but may not have realized were a key tool. Because frameworks are tools for organizing how you think of problems and a way to communicate about problems.
In every domain of expertise, experts use frameworks to organize how they look at the world. Frameworks help experts look at all aspects of the situation. And when provided information they can realize that they have not been given critical information that can change how they should approach a problem. Novices will often work only with what they have, like a classroom problem. I have noted that Generative AI foundation models make the same mistake, they work with only what they have in cases where a framework would have told them a direction to go to get missing information for a fuller picture of the problem.
The other use of frameworks is communication. Frameworks provide a way of quickly communicating the essense of the issue and the information that a decision will depend on. When I was deployed in Afghanistan, I had a brief that was working its way up the chain.It was being scheduled for a 2-star general. A couple of that general’s staff were present when I had given the brief at a lower level. And they told me that the brief was good, but I had to completely redo it to fit into a specific framework. Because that is how that general processed information. (In the end, the staff went through my brief and realized that the recommendations were sound and we got the result without having to formally present the brief)
What can a 5-hour class do? The Business Problem Framing course is a tour of a wide range of frameworks, probably familiar to someone coming out of business school, but not familiar to those whose focus is on analytical methods, math, and programming with data. And many of the frameworks are very similar in purpose. But the right framework for the problem is on partially about the particulars of the problem, it is also about the framework that communicates the problem to everyone involved. And just as having many ways of delivering a message is helpful to have, familiarity with a number of frameworks is helpful to have when communicating with stakeholders, because it is more possible that you will find a framework that resonates with everyone. And that leads to a better and more fruitful effort in solving the right problem at the right time.
Subject domains that lead to failure in large language models output
Published:
At the 2025 YinzOR conference I was talking with Léonard Boussioux about types of domains where large language models (LLM) have a tendency to fail, and other conversations encouraged me to write this down.
There are stories of the early days of aviation, where a test pilot would come back and learn that his plane had cracks, and were delighted because that meant that they were learning the limits of the aircraft. In that spirit we want to look for domains where the foundations models will give poor results, so that those developing applications can look for potential failures and design applications and train users to be attentive for errors. For this discussion, the cause of the errors are the data used to train the foundation models. Like other deep learning based models, to uncover categories of errors, we look at the training data.
Large language models tend to fail due to inability to work with nuance and naivete. My friend Polly Mitchell-Gunthrie describes LLMs as unable to work with context, collaboration, and conscience. I describe problems in LLMs as failures in nuance, naivete, and novice problems. Again, this is due to how the foundation models are trained (effectively all publicly available text), so these are social problems, and my not be solvable in this real of LLM based AI.
Novice problems are due to the characteristics of what is available on the internet. The majority of information on the internet is aimed at beginners. (computing topics are significant exceptions to this) So there is a lot of information that rises to the equivalent of an introductory sequence in college. So it has a body of knowledge. But using a body of knowledge that is targeted at introductory level leads to nuance, naivete, and novice errors.
Nuance issues are probably the most recognized. Nuance comes into play in subjects were details matter where the answer in a specific situation is not the same as the standard case. When given a setting, an LLM (like a novice) will take the information provided in the prompt and find other sources that include the same information and come up with an output (answer). However, and expert would take information and fit it into an applicable framework. Then, the expert will recognize that there is missing information that influences the final answer and ask for that information. Similarly, when considering other references, the same framework tells the expert the extent of applicability of that reference. An LLM only matches text in the prompt with the references, so will not always check that the context of the reference matches the context of the setting of the user. These types of issues lead experts to reach very different conclusions than people who are new to a domain, and the LLM tend to act like novices here.
As an exercise to help people identify domains where LLMs do badly, I ask people to pick a topic that they know well, but not through textbooks or classwork, and not computer related (this tends to lead to topics that they know experiencially or through true research). Most people identify a hobby, my manager did this exercise with his master thesis topic. Another variation of nuance are details that frequently occur together, but are not the same. Since the LLM works by probablisticly choosing words that occur together, it can often try to combine related topics or words that should not be. A frequent example of this is in anatomy, where LLMs trained on medical texts will often conflate the names of two body parts and into a body part that does not actually exist.
Naivete occurs when someone is in possession of facts, but does not recognize the consequences of those facts. For an LLM, it is easy to take a prompt, then from references that match that prompt, identify other facts/details that are typically associated with the information provided by the prompt. But unless it finds references that explicitly spell out the consequences of a particular collection of facts, the LLM will not provide the consequence. As an example, my then 10 year old daughter had written a story that was set in a domestic setting in the United States during the 1860s (U.S. Civil War era). So when I ran her through the exercise of a topic that was not well known, she asked the Generative AI about an aspect of domestic life, specifically methods for starting fires. Her comment was that the generative AI gave details that as far as she could tell were all true. But, it did not provide an important consequence. When given the same set of details, a modern day chemist would mentally translate the 19th century terms to modern day counterparts, and immediately recognize that it contains all the ingredients to cause an explosion. And in real life this is what happened so there are very few examples of this technology in museums, because they all exploded. And my daughter regarded that knowing a technology meant for use in domestic (home) life had a tendency to explode to be an important detail and the LLM not reaching that conclusion to be a failure.
Another type of novice error are exceptions and crossing domains. Many domains will teach general frameworks and rules of thumb at the introductory level. They are intended to help practitioners succeed and to avoid common pitfalls. However, past the introductory level practitioners learn the reasons behand the framework and rules, either from deeper training or through experience, so experts will know the exception to the rules or when to modify rules based on the particular circumstance at hand. This is even more important in cases where multiple domains are involved, which is common outside controlled environment such as academic or teaching environments. In this case, the standard rules for the multiple domains can conflict. Experts will resolve this both by establishing exceptions based on the circumstance, but also looking at the ultimate goal or intent of the activity, and break or bend rules based on which rules interfere with the goals or the mission. But they don’t completely through out the rules, experts will keep in mind the intent of the rule and ensure that the intent is addressed. When LLMs are given both the rules of the domains as well as history of prior activity, the LLMs will often identify the fact that rules are broken, and no longer follow the rules, which leads to poor outputs that do not respect the issues that arise with these domains in practice.
LLMs are especially handicapped when there are intersecting domains. When articles or other texts are written or published, the general rule is to have anything you write/publish be on a single topic, which makes it easier to identify the target audience and for the target audience to find your work. Topics that are within intersecting domains tend to be niche topics, and are both difficult to get published and difficult to find. An thus less likely to be included in the foundation models training data. Another area that is not found in published texts are failures. In many domains, expertise is developed through experiencing failures. However, these domains tend not to document or publish the failures that experts learn from because of potential of repercussions or public disapproval. And if these are not published, they will not be available for training foundation models.
The purpose of this exercise is to make Generative AI useful. And to be useful the ones who work with Generative AI models have to be able to recognize and look for so that they can screen Generative AI output for other types of errors. For example, my now 11 year old daughter continues to identify errors in Generative AI output ranging from trivial to profound, and because she has this ability, I have no concerns about her use of Generative AI. Same with my colleagues, once they have experienced identifying errors in AI (and this holds for machine learning models as well), they are able to identify future errors and react appropriately, and not taking the outputs of AI as automatically true. And this leads to more productive use of AI.
Failures and how does it impact the quality of Generative AI
Published:
I gave a talk on Generative AI as one of PyData Pittsburgh’s monthly events. While the focus of the presentation was on demonstrating impacts of randomness on Gen AI output, during the discussion we talked alot about how we teach Gen AI a specific domain, and what makes a person an expert and can Gen AI learn those things. There were a few things about being an expert that will take a lot of work to replicate when starting with Foundation models, but one that stuck to me was the role of failure in learning, and how hard it will be to teach this to foundation models.
My friend Polly Mitchell-Gunthrie talks about Context, Collaboration, and Consciencience when talking of the limitations of foundation models. Context is a well known, discussed, and acknowledged issue in Gen AI that we address through variations on prompting and grounding. Conscience is both looking at issues in ethics but also mission. But collaboration is harder, because Generative AI does not have institutional memory. In particular, the memory of failures.
In American culture (which is where I am), we have a pressure to be perfect, and to make no mistakes and no failures. But in a wide range of domains where there are high standards of performance, there is a maxim that is some variation of “if you have not failed, you did not try hard enough.” But even in these communities, we rarely document these failures, this level of training is done person to person, with mentors/trainers/leaders who provide cover to try different things and tolarate some level of failure in the pursuit of excellence. But more importantly for this dicussion, this does not get published, because these communities are cognizant of how intolerant of failure the general population is. But that means that the general population does not realize that the performance and excellence was developed through experiences of failure. And the lack of documentation means that foundation models do not learn this. (for a counter example, look at baking websites that explain causes of failure using pictures of baking disasters)
Instead of reality, the internet is a record of successes, and not failures. This is a known problem (it is frequently discussed in academia, with journals only publishing successes, without providing lessons learned from failures, leading to a lot of wasted effort as research groups go down dead ends that other groups had already explored.) But with foundation models, that means they are trained on the successes, and not the failures. So everything seems easy, and the Generative AI that uses these foundation models provides answers with assurance, but the people who have to implement them run in to all of the myriad of problems that come when doing things in real life.
Could you address this through grounding? This is a cultural issue, you would need to have a record of failures, where those who went into the unknown areas of your domain were allowed to fail without adverse consequence. Then you could potentially have the Gen AI realize that a path of action could lead to an unresolved problem. And you would have to accept the Gen AI discovering those failures, and actually telling you about them (things like this are part of the problem Gen AI has with understanding context). So, similar to problems where there are multiple correct answers, this is as much a cultural problem in what we as a society see fit to write down (which becomes part of Foundation models), and what we do not.
Adventures in core.logic: learning clojure and logic programming with help from Gen AI
Published:
This past month my project has been to learn logic programming, and as a vehicle to do this, learn clojure (again). For those who are not computer scientists, logic programming is one of the four main computer programming paradigms: procedural (what most people learn in an introductory programming class), object oriented (what most computer science programs and professional programmers aim for, Java, C++, C#, Ruby are all examples of OO languages), functional programming (Lisp and its relatives), and logic programming. The closest most people get to logic programming is SQL, which is declarative and works by expressing the outcome, but not the steps to get there. The most well known language is Prolog. A more recent expression of logic programming, is miniKanren, which is a Domain Specific Language originally implemented in Scheme, but there are other implementations, whose quality seems to be related to how well functional programming is implemented in those languages. This essay looks at (1) learning clojure (a Lisp that runs on the java virtual machine, (2) learning logic programming (3) learning core.logic, which is the implementation of miniKanren on clojure, and (4) using Generative AI to help with all these things.
This is my second exposure to Clojure, which is a Lisp (a functional programming language) that runs on the Java Virtual Machine. The big draw is that it provides a functional programming way of working that allows use of all Java libraries. As a data scientist, the advantage of functional programming is that this is a much better style of programming when doing data manipulation. For example, using R with the tidyverse is functional style programming in that you perform operations on data frames that return data frames, and this allows the use of piping/sequencing of functions that conform to this pattern. (Pandas in Python is a flawed version of this as not all functions in Pandas follows this rule)
My first run with Clojure was around 2014 (so says my Github timeline). At the time the Incanter project was trying to establish it as a data analysis environment on the JVM. With the goal of being used in corporate IT departments that had standardized on the JVM (which places obsticals to using Python or R). And it was good enough that I had written a model and associated analysis in Clojure for an attempted startup (a clean implementation which was not done at any of our home organizations). But the Incanter project stalled. And more recently a broader effort to provide data analysis/scientific computing capabilities into Clojure shows promise. Scicloj. One standard mantra that I can confirm. Lisp makes the claim that it has very little syntax, it is easy to learn. And I would agree. After almost 10 years, a short online course and a review of some books I had from 10 years ago I was pretty up to speed. Because when everything is a list, the question then becomes what is the form of that list for the task/function/library at hand. Which is easier than any other language that I work with where I have to learn the philosophy of every package I use. (or collection in the case of the tidyverse on R). In addition, the tooling was easier. Visual Studio Code has the Calva extension, which makes working with Clojure projects automatic (pretty much anything on the Java virtual machine needs an IDE to handle the project setup, so a good IDE is essential.)
For learning logic programming, I started with some Prolog materials, because that would allow me to focus on the logic and thinking part (Prolog is also fairly sparse in syntax). I got Adventure in Prolog by Dennis Merritt and followed along with implementing the Nani adventure game as well as the geneology exercise that was developed over the entire book. But I was always going to move to miniKanren, becuase in any conceivable use, I would be integrating logic programming into something else.
My first two attempts to moving from Prolog to a programming language were with Julia and Clojure. With Julia, there was Julog (which is attempt to follow Prolog patterns but in the Julia language). This seemed servicable, although all I did was the adventure game. Then I looked at the miniKanren projects. All of them were the beginnings of an implementation, but not complete enough to do anythihng. (Scheme and miniKanren both have a reputation for being the target of a budding language creator’s first target because they are so simple to write, but then the said creator’s attention goes somewhere else). And even though I have also used Julia in the past, I basically had to learn it over again as it changes every version (I review books by computer publishers, so I have had a chance to look at Julia every now and then, and it does feel like I’m starting over again every time).
Clojure has the advantage the the main language is very stable (and since it is a Lisp it has the advantage of having seen the history of language decisions, good and bad). They have a fun graphic where the show the history of the source code changing which looks like layers instead of comparable graphics for other language projects that look like landslides. But the same cannot be said about core.logic. When core.logic first came out it was a unique in the sense that it was an implementation of logic programming that was in a relatively mainstream computing environment (because logic programming makes a lot more sense on a Lisp type programming environment than on a Algol type object oriented/procedural programming environment). So there are a lot of early tutorials. But around version 0.8.5 or so there was a major change in the core.logic library organization, and a sub library was created to hold all of the non-logic things. Which includes things like facts and data. But this broke all of the tutorials. And like faddish things, noone updated their tutorials. So all of the tutorials that everyone points to was from 0.7.6 or so. So as I repeated the Adventure in Prolog exercises, the getting started introduction was easy, but I had to discover that there was a new way of doing things that involved actual data (as opposed to being logic exercises) and I redid the Nani adventure and the bird expert system using the new core.logic and core.logic.pldb structure.
The bird expert system exercise was particularly difficult. I actually did not do this set of exercises when I went through the Adventure in Prolog book (because it did not actually start until about halfway through). So I tried to start from someone else’s Prolog solution. And that completely failed. So I used OpenAI’s ChatGPT and Google Gemini to help me. So neither of them completely got it right, but they got me on the right track. So my solution does not look anything like the Prolog solution. And the types of mistakes that the Gen AI did were interesting.
Generative AI works by going through the training data (essentially the internet), and using the tokens (roughly a word, sometime part of a word and sometimes a phrase) in the query, identifies other uses of that set of tokens and comes up with a probability of options for the next token. Then chooses the next token randomly based on the calculated probabilities. Then, including the token the Gen AI just added, repeats the same and get the next token. And repeats. The randomness is what gives Gen AI its creativity instead of just being a search engine. But it also leads to mistakes, as the Gen AI does not actually understand any of its source texts, so it does not recognize the context of its sources or the fact that some sources may not actually go with others.
This gets more problamatic in a subject like core.logic, where the majority of the texts on the internet are out of date, in a breaking way. Normally I say that Gen AI is particularly good at computing related topics, but that is because of the vast quantity of material available on various message boards programmers and computing professionals frequent to ask questions and get them answered. Clojure core.logic is very different, as there is not much material (Clojure is not one of the more common languages, and logic programming is also a small niche), and there are at least three different eras, which are not mutually compatable. And since modern examples do not overwhelm historical ones in quantity, things get mixed together.
Now, how big of a problem is this. In my experiences using Generative AI to aid in programming (again, I am a data scientist, so I am interested in data type issues), Generative AI is good for giving programming structure and style (which is very useful, (re-)learning new APIs is time consuming), but it regularly gets logic and the model wrong. But as a scientist, logic and the model are things I am good at, so I don’t mind examining code to correct the logic and model, I wanted the help in getting the thing into a running state! This is why despite Microsoft reporting 40% error rates in Copilot generated code and OpenAI reporting 70% failure in software engineering project when using Gen AI, professional programmers still find Generative AI to be very useful. It does get things like how to work with an API right, and has pretty good programming style (with appropriate commenting!) But logic, which the Gen AI gets wrong, is something that any competent programmer does not mind doing themselves.
The key for using Generative AI is the same as other things. It is good for style and structure. Not so good for facts and logic. But that is what subject matter experts are good at. (and most subject matter experts are not so good at style and structure) So a trained SME can play to a Gen AI strengths and deal with the weaknesses. But only if the human is paying attention to this.
Next steps, repeating the Adventure in Prolog exercise, but using the Kanren library in Python,
Why take opportunities for public speaking as an analytics professional
Published:
For many of us in technical fields, public speaking often feels like a skill we left behind in school or perhaps dusted off for job interviews, especially if our roles involved training or teaching. Once we’re in the professional world, the focus tends to shift solely to our day-to-day tasks, and public speaking opportunities seem to dwindle. However, effective communication is crucial for professional growth, and unfortunately, workplaces don’t always provide sufficient feedback on technical presentations.
This is where engaging with local professional communities can be incredibly valuable. While I’ve had the privilege of speaking at professional society conferences, I’ve also found immense benefit in giving talks within local technical organizations. Many metropolitan areas are familiar with these as “Meetups,” named after the platform that serves as their online home. These local speaking engagements offer distinct advantages compared to academic talks or large industry conferences.
Low-Stakes Practice Environment
One significant benefit of giving technical talks locally is the opportunity for low-stakes public speaking practice. These communities are typically smaller, comprising individuals genuinely interested in professional development. Because many members also use these meetings as a platform to share their own insights, the environment is inherently supportive and sympathetic. It’s a space free from the competitiveness that can sometimes arise when individuals are trying to build a reputation or feel they’re in direct competition. This fosters a very friendly atmosphere for honing your presentation skills, where attendees genuinely want to see you succeed.
Sharpening Your Communication
Secondly, preparing a talk for a public audience compels you to think critically about what truly matters. In a work setting, it’s easy to gloss over foundational concepts because everyone involved in a project is assumed to have that background. In a public forum, you’re required to identify the essential information and ensure you cover it as necessary background. This is particularly true for work-related topics when you might need to use public datasets (as most companies don’t permit the use of proprietary data for more informal talks). This process forces you to consider what’s important for your audience and what’s technically crucial. It’s an excellent exercise in organizing your thoughts and effectively communicating them, a skill that translates seamlessly back to your work when you realize not everyone on your team has the same background knowledge.
Building Professional Community
Finally, these local groups are instrumental in fostering community. Recent articles in local Pittsburgh publications have highlighted the increasing difficulty of forming social connections after school, and professional colleagues, while valuable, often don’t entirely fill this gap due to shorter average tenures at companies and the inherent limitations of work-only relationships. Professional organizations offer the unique advantage of being specific enough to align with shared interests, yet broad enough to expose you to ideas beyond your immediate work. Giving a talk provides a natural reason for others to engage with you, sparking discussions and building relationships that can extend far beyond any single job.
(DRAFT) What do university departments provide to the employers of their students (data science)
Published:
I gave a talk at the 2025 INFORMS (Institute for Operations Research and the Management Sciences) Analytics+ conference (i.e., industry practice focused as opposed to research focused) on Where Should the Analysts Live: Organizing Analytics within the Enterprise. The talk was a result of many organizations asking if analytics should be managed within companies centralized or de-centralized. One of the topics that came up is the fact that much of the practice of data science is learned on the job. For some people, they may ask if this is the job of universities. I would argue that the practice of data science is so large that this is an impossible ask. I do so from the perspective of someone who for a while was an industry focused professor within an R1 engineering department.
First, what is data science? Drew Conway still gives the best definition that I have seen in the data science Venn Diagram
Math/stats are the full range of analytical methods as well as the scientific method (the ‘science’ of data science). Hacking skills are the computer programming, software engineering, and data engineering specific to working with data (as opposed to what is generally emphasized by academic computer science). Substantive expertise is the subject domain of the work, but it also includes the specifics of the company such as understanding its markets, its customers, and its strategy.
Math/stats is in principle the domain of our university departments. But university departments are specialists (and research faculty are hyper specialists. There are two problems with expecting university departments to cover the full range of math/stats that may be needed at a particular company. First, university departments focus on a particular domain, so it is not expected that they cover other areas of data analysis that a company may need based on their particular interests. Second, they have limited time and unless you are at a very large state university with a particular mission to cover the full range of a subject area, the faculty of a small or medium size department cannot cover the full range of topics that are associated with a given field of knowledge. So departments create undergrad or graduate programs to cover a foundation, then allow students to specialize (in areas that the department can cover with the faculty they have). As a non-tenure stream professor, I would explain to students that departments hire to cover a wide range of their field, so they generally do not have much duplication. But each department has to make a conscious choice for what they cover and not cover every time they make a hiring decision.
So what is a university promising with their graduates? The base set of knowledge and methods (and methods are more important than knowledge, because it is easy to refresh knowledge, you actually need practice with methods), for STEM (and social sciences) the scientific method that creates understanding through iterative experimentation and statistical analysis of experimental results. And most crucially, the capability of learning a technical area. This ability to learn is arguably the most important part of this whole exercise. Because the world is a big place, and a 17 year old high school student will not be able to predict what the next 40 years will be like. So where a 22 year old college graduate is capable of will be nothing like what she will do over the course of a career. It is hard to develop this ability without college. High school tends to be focused on what you know. And it is too easy in most jobs to just do what you are doing now, unless you already have the experiences of having to learn new/different domains. For example, in most STEM and the social sciences, statistics is a side knowledge domain. But for those who go into data science, the fact that they learned statistics makes learning applied machine learning easy. And the scientific method, while it may not be the thing you think about when you think about engineering or economics, is ingrained into the methods by which they see the world. It is relatively easy to teach skills, it is much hard to teach mindset or the ability to learn new ways to think.
Is there anything different about artificial intelligence? Actually, yes, which makes it easy to learn for STEM and social science trained people, but also dangerous. By definition (see Section 238(g) of the National Defense Authorization Act of 2019) any version artificial intelligence are those which perform tasks without significant human oversight, or that can learn from experience and improve performance when exposed to data sets. In particular, it means that the creators of an artificial intelligence system or model do not have to know how the system that the AI is being added to works. For those in the mathematical sciences (e.g. mathematics, statistics, applied math, operations research, computer science), this is incomprehensible. Even the most theoretical researcher has a core belief that any application of mathematical models involves representing important aspects of the system in mathematical form. But this makes AI (such as machine learning) relatively easy to use in practice, and this has a low barrier to entry. But if someone, like a company, actually has subject matter expertise relevent to the problem at hand, not incorporating that expertise into the model is lost value.
Is it enough to be able to learn new skills as needed? No, we also have to be able to learn to think differently. The most prominent example is Generative AI. For those who only have knowledge and skills, Generative AI is a completely new thing. For those who are able to come up with new ways of thinking, Generative AI a combination and extention of deep neural nets, natural language processing, and reinforcement learning trained on the published internet. And its strengths and weaknesses are not random facts akin to gotchas, but are based on characteristics related to its origins. And knowing that makes a world with Generative AI different, but something that we can use. This past week I went to a seminar on quantum computing. The mathmatics are completely beyond me. but I could understand enough to recognize the reason for its promise, what is lacking, and some sense of what are some key intermediate steps that have to happen if it ever reaches the promise that many talk about. And this practice of being faced with completely new subject domains is something I do frequently.
So what can companies expect from the graduates that come from their university partners (whether through former relationships or merely through hiring in the community). Sometimes it is a collection of specific skills. But more important, a college graduate comes with a testiment that person is able to learn a range of skills and knowledge that are part of a cohesive whole and put them to use. And having done so once, will be able to do it again over a 40 year career.
On simplicity in data science communications
Published:
Everything should be made as simple as possible, but no simpler paraphrase of A. Einstein
I spent this past week at the 2024 INFORMS Analytics conference. One of the major themes across both the speakers, the panels (of which I was a panelist), and conversations in the halls was soft skills for data analysts/Data Scientist. Data scientists are subject to the same stereotype that is used with all those who are technical specialists, that they lack soft skills and this prevents our end stakeholders from understanding and being able to take advantage of the knowledge and capability we bring to our organizations. And that the most commonly provided solution is to ask the specialists to simplify the delivery. But, other than an excuse for those with business backgrounds to beat up on the geeks, I don’t know if this is the right solution. A better direction would be, as John-Eric Bonilla described it, the data scientist acts as a translator, the person who takes the aggregated insight of the data and of the subject matter experts throughout the organization and translate that into the framework of the decision maker. This is a tall order, but this is the reason that Drew Conway in his Data Science Venn Diagram gives subject matter expertise equal weight with the math/stats and the computer skills that get so much prominence in these discussions.
When I was deployed in Afghanistan, a brief that I prepared was being pushed up to the Commanding General, ISAF- Afghanistan. Members of the General’s staff was present at the last brief, and their comment was that it was a good brief, but I needed to redo it in their format. And they gave me a highly specified template. Now, I could be judgmental and say that it met no conceivable definition of “simple”, but I won’t. Because I realized immediately that the template had two functions. First, to shortstop any Powerpoint Rangers and save the General from Death by Powerpoint, because a commanding General in a combat zone is a busy person and does not have time for that. But also because the purpose of the template was to present the information inside the framework that that particular General processes information for the purpose of making a decision. And that specificity of presentation, that the recipient can process the information using the framework they have as an expert in their area and make a decision, is the goal of technical communications, such as data science.
The common recommendation to data scientists is that we need to simplify our work for presentation to our decision maker audience. And the reason that we are given this message is that our decision maker audiences do not need or want our technical explanations and they cannot understand complex topics. But this view of our decision maker stakeholders is demeaning. So far, in my career I have found the decision makers that motivate my work to be intelligent, subject experts in their own right, and fully capable of understanding detail and nuance. But the key is not to remove subtlety and detail (which is the reason this person is in the position of decision maker), but to present the subtlety and detail that is important. Certainly, the tendency of technical experts to want to focus on the story of their work does not help either. The answer (IMHO) lies in the use of frameworks. Every specialty community that I know of has frameworks that are used to organize and communicate information. Examples are the range of SITREP formats used by specific emergency response and military communities, the 9-line medevac report, frameworks used for reporting patient condition in specific circumstances in the medical community. And individual leaders have developed a framework to make the decisions, even if this framework is masked in intuition. And in the ideal case, that record of good decisions is the reason they are in their position.
That makes the key of data communication is to understand the decision making framework used by experts in this situation. In the case of the then Commanding General - ISAF, this framework was formalized by the General’s staff, so that all briefs going to him were presented in that framework. And the General can fit all of the provided information into his internal decision making framework.
When this framework has not been formalized, the key is direct communication between the data analyst and the decision maker (or surrogate). The data scientist needs to communicate with the decision maker, or someone who knows how the decision maker things (either intuitively, or because they are members of the same professional community who analyzes information in a standard way) to understand how the decision maker thinks. Then, this identifies both the type of information and the criteria that will be used to make the decisions. The data scientist task becomes either identifying the data needed to provide this information, or to use the data that is available to come as close as possible to the information required. And this unlocks the value of the data scientist, without diminishing either the role or capability of the decision maker.
There are a few ways for this to fail. First is from the data scientist side. Many technical experts have no desire to learn the decision maker process. This is often accompanied by beliefs that the technical facts make the needed action self evident. Then from the other side, there are those who think that they give commands to people and the people doing the work should be able to get it done without needed resources. Both fall under the heading of lack of communications between the analyst and the customer, which is universally known to be the most common cause of data project failure. The role of the data analytics manager is to ensure that constant communications is maintained and to intervene if not. (There are managers who think their role is to be a broker. But this also breaks communications and does not help change the most common cause of failure in data analytics projects)
Is being a translator easy? No. But I have found on my projects is that the data scientist is often the first person who realizes all of the people who are actually involved in an activity, because the data scientist is tracing all of the data elements. So the data scientist needs to learn everyone’s language to get a good picture on what is actually happening, and communicate to the decision maker in a format the decision maker can understand. Yes, this is hard (80% of data projects fail, and while vendors use this to market products, those who investigate that number say it is mostly communications). But we are not the only people who have to take complex information and transmit it to a decision maker in a form they can understand and use to make decisions. The UX community does this too. And, often they do it well. So can we.
Courage in Thirteen Lives
Published:
I paid attention to the Tham Luang Cave Rescue in 2018, not only because it was in Thailand, but also because of my background in both emergency response and in logistics (which was a major consideration in the rescue operation). I used this as a white board exercise in my supply chain and logistics classes as a professor. And a regular reminder for me is that I use the Saman WOD (created by Crossfit Chamgmai), which memorials Saman Kunan, a Thai Navy SEAL who died in the line of duty during the rescue, annually as my birthday WOD.
Thirteen Lives is a documentary style movie about the rescue, told from the perspective of the cave rescue team. With the access and active participation of the cave rescue team, it goes deep into the decision making and ethical dilemmas that those who were responsible had to address.
The choices in the cave rescue are well documented. There was the local team, the Thai Navy SEALS. Well motivated and trained, with the logistics planning expertise of a U.S. special forces unit to back them up. But, their training and expertise where not in the hyperspecialized setting of cave rescue that they found themselves in.
The choices of the cave rescue divers was also well documented. The need to work within the local system, which meant both the Thai civil and Thai military authorities (who were competing with each other in the way things things go anywhere in the world). The difficulties in just finding the team in the dark and flooded caves, then planning out how the boys would be brought out of the cave, with the host of ethical dilemmas that various aspects of that operation entailed.
But Thirteen Lives presents a profile in courage in the person of the governor. Thirteen Lives presents him as a governor who is in his last days, and implies that it was not a glorious send off. And that he was the apparent designated fall guy if things went wrong (with the foreign cave rescue divers being not too far behind, but that is a story that everyone is happy not to have to tell).
The first choice he was presented with was to work with the foreign cave rescue divers while in the presence of the Thai Navy and SEALS who were eager to do the job. On one hand, a provincial governor recognized that high profile foreigners dying looks bad to the Thais (who would be suspected of using amateurs and admitting lack of confidence). On the other hand, the loss of one of the Thai divers in the rescue was an indicator of just how dangerous and difficult this was. Further tough choices came when the boys were found, and the cave rescue team provided very honest assessments on the options they had and their chances of success (noone thought their odds were good, and almost guaranteed fatalities). And as the senior authority, he had to give the yes or no every step of the way.
But the most telling illustration of leadership in action was when the hydrology people explained what was happening with the rain on top of the mountain. That water was seeping into the mountain and into the cave system (which is what happens in all mountains, this seeping happens in the hills leading into my backyard). And that it could be possible to divert the water falling on the mountain so that it did not go into the cave system. But “there was a price”. The water had to be diverted somewhere. So the governor had to go to the villagers to ask to flood their fields in the name of a rescue effort that noone knew if it would succeed. So the villagers are private citizens, and there is no law that says that the governor can order them to take the damage, and certainly not for a mere chance of 13 lives (compared to the ruination of the villages that would be a guarantee) So, with no time, he has to persuade the villagers to do this, with only promises that he will try to make things right in the end, succeed or fail (and remember, he is leaving his position when all this is over) And this is the result of trust and leadership gained over time.
There are many images of leadership. An alternative is to cut your loses and avoid the possibility of things that don’t look good. Which often leads to loosing everything for no gain. And that was the alternative provided to this governor. But leadership is shown when you are willing to make difficult decisions. And convince people that it is everyone’s best interest that they sacrifice, on nothing more than a promise that they will all gain in the end. If not materially, then spiritually. And Thirteen Lives tries to show this in a very real way.
Theatre during COVID-19: The Catastrophist by Lauren Gunderson and the Marin Theatre
Published:
Live artistic performances is something that has been impacted by the COVID-19 pandemic. We have somewhat filled in that gap in our lives by watching performances by the Pittsburgh Ballet, the Front Row series by the Pittsburgh Symphony, and embracing the online performances by various artists (and paying for them). Because, just like we do with restaurants and other businesses, life just makes much more sense when we take advantage of the experiences that are available instead of dwelling on the experiences that are not. (and not being in the privileged upper 1%, the peak level experiences like being close enough to see the eyes of the performers that are the hardest to replicate with social distancing were not available to us anyway so these virtual performance give a pretty good experience)
The Catastrophist is a one man play by Lauren Gunderson, commissioned and written during the COVID-19 pandemic. And while many writers try to research their works by immersing themselves, Ms Gunderson was not able to claim COVID-19 restrictions as an impediment. She is married to the virologist who is the basis for the play (Ms Gunderson is known for writing plays centered on scientists, but she usually does not have this close access to them).
Dr. Nathen Wolfe is known as a virologist who achieved some notoriety during the 2014 Ebola outbreak which was more widespread than the normal and threatened to spread into the U.S. He is one of the virologists that persuaded then U.S. administration to take this seriously and prepare for its arrival. (I remember being taught procedures we would take in my role as a Red Cross volunteer if it was necessary to support Ebola quarantine by delivering food.)
There are two threads in the play.
First is Dr. Wolfe’s development as a scientist. There is an discussion woven throughout with his development as a scientist (Jewish grandma: “Why would you go to Harvard Medical School if you were not going to be a medical doctor?”) and a developing conversation on philosophy of science. But the meat is doing virology, especially in his specialty of zoonotic viruses, viruses that jump from animals to people. He talks about doing field work in Camaroon (which was partially under Dr. Donald Burke, of the University of Pittsburgh Graduate School of Public Health) going into the countryside and working with people living there and both teaching best practices and doing research with both the people as well as the animals they hunted. And how this field research led to the discovery in the lab of proof that viruses jumped from animals to people. (and for those who are linking this to COVID-19, this is the same work that the Wuhan virology lab does in the forests of Asia, they can discover a virus in the lab from sample they take, but any virus they discover is already existing in the forests and caves of Asia). Then, after going from field work to lab work, how this naturally grows to the insights that showed that the 2014 Ebola outbreak was different and more of a threat to the world than earlier Ebola outbreaks, and led to the warnings that the rest of the world prepared for it and was ready for it when it arrived outside of Africa. (there is a sidenote here about the accusations that occur whenever anyone actually tries to do anything in the real world saying that the response was botched because of it was not perfect. But after 2020, we know know what a botched response looks like, i.e. what would have happened if his team was not involved)
Another major thread was about risk and mortality. In part the conversation about risk was about philosophy of science, but then it becomes personal. There is an abstract conversation about risk, like he promised his wife that he would not do anything that exceeded a certain value on a risk measure. But then it gets more personal. Like the reality that every male in his family dies in their 40s from a heart condition. So it covers his relationship with his father, and then with his sons. (and that his sons never met their grandfather, who died just before the first was born).
Watching this with family (and after watching Dr. Wolfe’s TED talk), we have to remember that theatre requires its actors to exaggerate emotion because voices and body are all they have to communicate, especially in one-man shows where there they cannot use the reactions of the other characters (my wife, who does not like musical theatre, finds this comment amusing). So the play is a caricature of Dr. Wolfe. But is it a very good picture on what it is to be in science, the joys of discovery, and the thrills and hazards of actually doing work that has impact, especially in the politicized world of real life where there are plenty of people who want to blame those who save them. Highly recommended as a way of understanding how viruses and pandemics work (and the subfield of zoonotic diseases, think how hunters interact with the animals they hunt)
The Catastrophist is streaming on demand at the Marin Theatre through the end of February. Note that there is a one day delay from when you order tickets to when you get the link, so plan accordingly. For those with young children, note that theatre requires actors exaggerate their emotions, so there is cursing in anger sprinkled throughout the play.
Book review: Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy by Cathy O’Neil
Published:
Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy by Cathy O’Neil
My rating: 5 of 5 stars
There is a popular diagram that describes data science as a combination of math and statistics, computer programming skills, and subject domain expertise; and describes the dangers of what happens when one of those three are not available. But among academics, there is an opposed line of thought that says that math and statistics methods are pure and subject independent. This book is firmly against the idea that algorithms are a defense against bias. The reason may be that while the mathematician/machine learning modeler may be naive, the setting of the implementation is not, and the questions that are being asked as well as the data being used to train models are both affirmative choices where the analyst and customer have agency. And pretending to be a naive analyst leads to errors in the result that have real consequences.
O’Neil goes through a number of cases. But while many accounts will go into the “evils” of big data and machine learning, she does suggest good practices that can prevent the dangers. First evaluation of the model. The model should be tested by actually looking at its predictions and seeing if they are true. In statistics this is done through control groups. In data science this is done through holdout test sets. And in her case studies, she points out this is not done. Next, compare the model input data set to the population that will be applied. Again, she regularly points out where this is not done. A third one is the well known make sure that you are not using machine learning to perpetuate an undesirable status quo. (but this argument is too easy)
Read this at a book club at work. It spurred a great discussion, that has carried over into other conversations. Definitely recommended to those involved in data science/analytics where people are impacted, and in areas where data analysis is becoming a bigger part of life, so as someone who works with the results of analysis you can ask good question, both while the analysis is planned and performed, and in understanding and questioning the results.
Senior capstone during COVID-19
Published:
This is the month of COVID-19. Both University of Pittsburgh and Geneva College (our employers) announced that they would be delayed the week after spring break on March 11 (pushing back opening until March 20. The Commonwealth of Pennsylvania declared that all educational institutions be closed on March 17. Which also meant that our kids were not returning to school.
For looking ahead, I had told our senior capstone course during their midterm presentations on March 6 that COVID-19 was coming, and that they should expect it to be here before they returned from spring break. Chatter among the Chinese community in the north hills of Pittsburgh was about when to pull the kids out of school, first in dread of the return of college students from spring break, then after that watching COVID-19 march toward us when it reached Cleveland. And we pulled our kids Friday March 13. That day, our school district announced that everyone should be prepared to not come back to school from the weekend, somewhat jumping the gun on the Commonwealth’s announcement, but basically being correct.
All the schools (universities and kids schools) went on a one week pause while all of the teachers figured out how to convert their classes to online for the foreseeable future. For me, this first affected the Pitt Industrial Engineering Senior Capstone course, which had corporate sponsors. Once Pitt announced the delay of school, I sent a message to all of the students:
As you have heard, the University of Pittsburgh is postponing the start of classes until Monday March 23. The extra week is to give faculty a chance to make a plan to move to online delivery of classes. While the university and school of engineering are working on setting policies and plans, I can offer the following:
Note that while the classes are suspended, society as a whole, including your clients, is moving forward. Please contact your sponsors and discuss working arrangements.
For team meetings, have a discussion on how to hold these as many students may not be on campus. I recommend the use of collaboration software. For my own work, I am planning on using Microsoft Teams and I recommend it as one alternative. For those who do not know, MS Teams is a collaboration suite that comes with Microsoft 365 Online (those of you who use Outlook online or MS Office online with Box are using 365 Online) Information is located at https://www.technology.pitt.edu/services/microsoft-teams It includes text chat as well as video chat, screen sharing, and whiteboard sharing.
After you have determined how your group will work, please contact your faculty mentor and discuss how you will continue to work with your mentor. Note that faculty are still available to assist.
One major issue that was highlighted in the methodology presentation was how to work with the lack of data. You are encouraged to discuss this with faculty (subject to your mentor). In particular, the field of input modeling within simulation includes methods for working with limited data (I have discussed this with many projects, and the long term feedback from the clients is that the resulting models have been useful). In addition, many subject areas have standard methods of soliciting expert opinion.
For end of term activities, there will be meetings at the school of engineering this week and next to discuss how these will be modified. I will inform you as they get made.
Specifically, was the need for a continuity of operations plan On March 16 I sent out:
Reminder to all groups.
- Please send me and your faculty mentor the report specified in 2 by the end of today. Also, let me know as much as you know about (3), (4), (6).
- You should have your next meeting with your mentor scheduled. As per recommendations, this should be a virtual meeting. Make sure that you have a videoconference method that you have complete access to (i.e. no time limits) (Note: as Pitt provides you with an account MS Teams is such a method. The faculty should be getting enterprise Zoom accounts soon, but that requires the faculty be present in the meeting.)
Professional Note: What we are doing is called a Continuity of Operations Plan (COOP). While not always done, there are many management recommendations that any organization have one that covers loss of key personnel, loss of access to facilities, and potential disasters. Ideally (but not always done) this is exercised on occasion. E.g. I volunteer with Red Cross for disaster response operations in western PA. Every now and then, we have an exercise where the staff are hands off, to confirm that the volunteers can manage a regional disaster response in it entirety without staff participation.
—-
The Capstone program coordinators had a meeting this afternoon.- The Design EXPO is cancelled.
- Please inform your sponsors.
I need a report from each of the teams by Monday to understand your status and resource needs.
- What is your plan on remote collaboration (again, I am recommending Microsoft Teams as a collaboration platform if you have not chosen one). Have you tested the collaboration platform?
- What is the status of team members (i.e. where are they geographically)
For teams that are not substantially done (substantially done means you are running models and analyzing results), what needs are remaining? What is your plan on meeting those needs?
If your needs are data related, you should anticipate the possibility of not being able to collect data. Do you have the expertise to work with minimal available data and generate appropriate probability distributions (Note: these are topics from probability and from simulation) Is your faculty mentor able to help you? If not, I (Luangkesorn) am the faculty with the most research and practical experience with knowing what can be done with limited to no data.
- Discuss with your sponsors your plan for ongoing collaboration. In particular, how the final presentation will be done. Note: Pitt has Microsoft Teams accounts for all of you (see previous message) and I recommend that as a presentation platform if they do not have one.
- As the EXPO is cancelled, your company presentation (if recorded) will be your final presentation. You should have the presentation recorded and have your faculty mentor(s) and myself invited. Check with your sponsor if there are any security considerations. If you are not able to make a recording or invite the faculty to your company presentation (or if there is not a company presentation before finals week), we will have to make another arrangement (i.e. a presentation over Microsoft Teams)
- For all teams, check with your faculty on if there are any limitations on your reaching out to other faculty and getting assistance. You must do this. In the past, there are faculty who have gotten very angry about their team consulting with other faculty, even at the point where the team was fearful of project failure. You need to have an affirmative response that consulting other faculty is acceptable with no reservations. Do not wait until you need help to get this taken care of. My instructions from the department are to ask for permission to respond to your request from help, and to allow the team to fail if the answer is no.
- As I mentioned on the first day of class, during senior design, life happens. Back then, I was thinking of individual events, but this is clearly something much larger. Just as I said on the first day, be flexible and take care of each other. While I asked about geographic locations earlier, also check on everyone’s physical and mental states. We are all human, and everyone’s situation, especially those at home, is different.
- If you have questions or concerns, or even if you need a human contact, contact me.
Over the next few weeks, as classes were in turmoil from uncertainty, a number of students were sending me notes of thanks for providing structure for how to work with all the turmoil. Many noted the slight irony that one of the most unstructured courses they had (capstones are supposed to be about the students taking on management roles and providing their own direction) suddenly became one of the most structured courses they had (my directives were to create a Continuity of Operations Plan, so they had a well defined task that gave the rest of their semester structure) while all of their other faculty were still figuring out what the rest of the semester was going to look like.
Film review: Meru (2015)
Published:
| Jimmy Chin leading on an ice path on the 2008 Meru attempt |
Meru is a documentary about three elite climbers; Conrad Anker, Jimmy Chin, and Renan Ozturk; and their two attempts to take the “Shark’s Fin” route on Meru, a 4,000 near vertical wall of rock that is the last part of a 21,000 climb up rock and ice. It is styled as more technically challenging than Everest, because it combines a difficult ice climb and a difficult rock climb. Anker and Chin are represented as climbers who have been on many expeditions before, Ozturk, while an elite climber, was the new person on the team.
The movie’s focus is on the two attempts at this route, one that has never been completed before. And in between there is much discussion of risk. The movie interviews the climbers, other elite climbers, and family members and they discuss a lot about risk. Family members talked about thinking of a sibling or spouse who could very well die on a mountain. The climbers spoke about other elite climbers they knew who had died on climbs, and some of those climbs were relatively normal.
There was a period of my life where I was regularly doing some activities that were inherently dangerous. And the possibility of dying was not something I cared about. I think at the time my view was that I was in the moment, and when life ended, it ended. I like a line from Conrad Anker, these activities are dangerous, but we approach them with skill and knowledge, and when we take risks, with do so knowingly and deliberately. And if we should have an accident, our thought is one of profound embarrassment, because it meant that we did not know that we had crossed the line of too much risk. Jimmy Chin had a line about the excitement of it all, the rush comes because we are doing something that is in fact dangerous. So it is different than say an amusement park, which has speed but is under control.The thrill comes from mastery, that danger is faced, examined, and confronted, and overcome, while always under control. And the glory (to the extent there is any) is from knowing that you could go up to that line, but always recognize it and back away before crossing.
The documentary is about two attempts 2008, and 2011. The first attempt failed after they lost three days in a snowstorm, and their limited amounts of food led them to turn back a mere 150 m from the summit. And this is almost an example of their skill, to have the judgment to turn back even when they were so close.
The film then talks about risk in another way. Chin and Ozturk suffered a pair of accidents while on a commercial shoot on a ski slope. Ozturk was severely injured, Chin was caught in an avalanche. So the lead into the 2011 climb was both of them recovering from this, both physically (Ozturk was nearly paralyzed in several different ways) and mentally.
What sold this was the discussion about risk. Climbers and other sports are often viewed as crazy daredevils looking for thrills. There is a thrill, but it is not from the excitement. The thrill comes from bring to bear knowledge, skill, and judgment in an environment were all three are needed, and surviving in the face of real danger.
| Conrad Anker on the “House of Cards” pitch |
Using an Adafruit Circuit Playground as a Makey Makey fruit piano
Published:
One of the inspirations for the Adafruit Circuit Playground (CP) was the Makey Makey (http://www.makeymakey.com/), which provided a way to use touching conductive objects as input to a computer. Because of this, Makey Makey is commonly found in schools and maker faires as a means of connecting the world to electronics.
Because the CP has 8 capacitive pads, it should be possible to do something similar with the CP. Using the serial monitor output, we can see what the CP registers through each pad, and set thresholds for an action, for example, lighting up an LED or playing a tone.
The first version of the Makey_cp used the Capacative touch pads directly to drive the LED and the speaker. By observing the capicative touch pads on the serial monitor we can see that the steady state value of the pads was between 1 and 10. Touching the pad increased the value to around 30, but often an adjacent pad would also increase in value as a finger affects the electric field around the adjacent pad. So we set a threshold on the CP capacitive to 20 for use as a finger piano. In addition, we use the switch to turn on and off the sound.
The sound is tuned using the pitches.h definition file, which has the tones for each potential note. As the CP has 8 touch pads, this is enough for a full octave of a major or minor scale.
To serve as a Makey Makey, the pads need to be connected via alligator clips to something that can conduct electricity, such as fruit or a metal conductor. Because such objects have capacitance, the noise level on the CP touch pads increases. In order to increase the signal from touching the pads, the person touching the pads should be connected to a power source, in this case we use alligator clips to the 3.3V out pad of the CP. This leads to the CP to detect a value of approximately 150.
Next, we attach alligator clips to the eight capacitive pads and to fruit. This creates a fluctuating reading around 20. We find through the serial monitor that touching the fruit while connected to the 3.3V out leads to a capacitance reading of around 225. The threshold is then
My black belt journey
Published:
There is a cliche that the path to a black belt is about the journey, not the destination. And that is true for anything that takes multiple years to complete, because anything that takes years is no longer an impulse decision.
| Self defense testing at the ATA Taekwondo |
| Elbow strike drills |
My fitness journey started when I was young. I was always one of the small kids in the class, which is a problem in the organized sports that phys ed was centered around in those days. But, even then, my phys ed teachers made the comment that while I never was strong or skilled, there were many points where I made up for it by shear effort. College was marked by suddenly realizing I was getting winded, and I started running then. After college when working in DC, I continued running longer distances, with 1/2 hour and hour long runs being weekly events. (my roommate also did this, of course, we were completely overshadowed by his then girlfriend). I continued running and hiking in grad school, and started 5K and 10K races. Once when backpacking, I realized that if I was spending days going up and down ridgelines, I could certainly pull of the five hours of running needed for a marathon. So I ran 1/2 marathon, then my first marathon (which was just under 5 hours). I continued this after grad school while working. And by now I have six marathons (Chicago; Ottawa, Canada; Philadelphia; and Bagram, Afghanistan) along with numerous half marathons and three Rachel Carson Challenges (34 miles hike in rugged terrain in daylight)
But reality, especially when kids entered the equation, was that my running had diminished. I was no longer the neighborhood crazy who was running in all weather conditions. My weekly long runs were not much longer than my daily runs from when I was training seriously. And we were concerned about my four year old son (who gets to tell his own story).
After my son started taekwondo, we did the first month, then six more. As we were thinking that this was something he could stick with, I decided to start as well.
| Self defense testing at the ATA Taekwondo |
| Self defense technique testing, with the school head instructor as designated attacker. For this particular rank, all the self defense techniques learned since the beginning were practiced in sequence, which made for a very entertaining testing session. |
At the beginning, I did not join my son. And my reasoning is that there should be something in his life that was apart from us (parents). But a few thoughts led to changing that idea.
I was confronting the reality that I was not a regular runner like I was. And the days when a 10K was easy were past, and 5K were now the long run, and I did not even make that every week. While I was never structured in the past, having a bit of structure in my physical fitness was probably a good idea.
I always liked the idea of having some area in my life that I was active in that I was not very good. I like learning, and this could be a new thing for me.
Because we were thinking of committing our son to years of future in taekwondo (and with my son being very enthusiastic about this prospect), I wanted to know what kind of kids my son would be learning from and looking up to. Because one thing about the structure of martial arts, in any reasonable school, the senior students end up being teachers and role models for the younger students. And since in this school teens and adults were together, that means I got to see the kids who were growing into the future instructors of the school as they grew, and I would get a look at what my son was going to become (this was a young school, so I was watching the first group of students reach the point where they could start teaching)
Did this work?
In addition to 2-3 classes a week (sometimes including classes my son and I attend together), I regularly train on my own (sometimes with my son), and we also do other fitness workouts outside the school (bodyweight exercises in addition to drills and practice).
It is not just learning something new, but it is also having someone pushing me on to be better, in the standard sense that the role of the teacher is to help you be better than you are today. And the art part of martial arts has certain appeal. The head instructor at the school calls me a technician, because of the way I work on forms/poomsae. (kata for the Japanese types)
And I do get a very close up view of the kids who are on the instructor track now as we train together in class, and I get to watch them help instruct the classes that my son is in. And confirm that they hold him to the highest standard that he can reach!
There are other fun moments. When we started, my son was higher rank, so I reminded him that the job of high rank was to ensure that the lower ranks were training properly. (later, we crossed over as at my school the younger kids take twice as long to advance in rank than not quite as young students.) At the family classes, my son and I would find ourselves in races doing fitness warmups (pushups, situps) with the pre-teens and teens. (implication is that I was faster (more fit) than most of the kids, so they could only catch up when during my son’s turn). And I like to joke that the adults in the class have to remind the teenagers that we can still keep up.
| Testing for black belt in ATA TKD |
| Sparring during belt testing |
Black belt testing was a show, as it should be. Board breaks with smoothness and power, and a poomsae that was crisp with satisfying pops of the uniform sleeves and legs on punches, blocks, and kicks. As it should be.
What next? I like the discipline of regular feedback, and martial arts (especially forms) has the mindset that there is a perfect form, that we strive for but never reach, that I find appealing from an artistic point of view. So I plan on continuing for the indefinite future. Would I go on? The standard pathways (as I tell people who are thinking of how this goes on college applications) are competitions, teaching, and judging. And the reality is that I have multiple avenues that my time could be applied in setting where I am in a role of leadership (formal or informal), so going down those avenues in martial arts is probably not the best use of my time. (other than it is a point of reference with my son for the foreseeable future)
In my fitness area of things to learn (since Black Belt means I no longer count as a beginner), I am starting to branch out into other areas of fitness. Many more forms of bodyweight training (which my kids are very happy partners), but also using resistance equipment such as dumbbells, suspension trainers, and resistance bands for strength training. Which means someday I will have to do more sessions with a trainer to get the most use out of this.
The numeric designation of a Black Belt is 1st degree. The karate term shodan also carries the connotation that attaining the black belt is merely the end of the beginning of the journey. And I look forward to the years to come.
| Testing for black belt in ATA TKD |
| Square block in Choong Jung 2 form |
Book review: Personal Memoirs by Ulysses S. Grant
Published:
Personal Memoirs by Ulysses S. Grant
My rating: 5 of 5 stars
The memoirs of Ulysses S Grant begin with him attending West Point, being a junior officer in the Mexican-American War, then his more well known role in the U.S. Civil War. What I found notable was his willingness to look at his actions and the actions of those around him and admit of what worked and what did not, and a fairly levelheaded and charitable view of those who commanded, served under, or opposed him.
The first part is a view of what the war looked like to a junior officer, prior to and during the Mexican-American War. It was a learning experience for him to lead men, and also look at the quality of the officers above him, who would be the commanding generals of the first part of the Civil War.
The Civil War section was a lot about managing logistics, and how logistics drove the campaigns. The actual battles do not as much discussion as the discussions of overall strategy and goals, and how logistics drove how he ran the various campaigns, and discussions of leading generals on both sides and his opinions on each.
The book is also notable because he discusses the backgrounds of both wars, and it becomes a touchpoint on what the country thought about as the cause of the Civil War in particular during and shortly after that conflict.
Definite must read into what being at war is like at the junior officer and at the general officer levels. And a 19th century feel of the U.S. Civil War.
Setting up the Raspberry PI with the 7 inch touch screen and SmartiPi Touch display mount.
Published:
I recently got the Official Raspberry Pi touch screen and the SmartiPi Touch display mount. The main reason I got it was that I was getting a couple things that are intended to work with a computer, and I wanted said computer to be portable. And the Raspberry Pi with Touch screen could be run off of a USB battery. And this also means I (or my kids) won’t be tying up a computer (or a desk) that I would rather be using for work while they do their explorations.
The standard references for assembling the SmartiPi Touch are the videos by the creator, Thomas Murray, and and a more complete video by Francesco Vannini (both videos embedded at end for reference)
However, many commentators have noted that there is not a good set of written directions, and I think the big disadvantage of an expert commentator is that they don’t always remember to say things that a non-expert may have a hard time figuring out. I mostly followed Francesco Vannini’s video, so here are a few additional notes.
Here are the boxes, with a Raspberry Pi. Not pictured is the Sense HAT that I will also put on.
All the components from both boxes. Note that not everything will be used.
At this point I have:
- Added rubber feet to the corners of the stand (back side of the part that is mounted on the hinges).
- Attached stand to case body using included nut and bolt.
- Connected the display connector ribbon to both the display driver and the Raspberry Pi. I used the one that came with the touch display. I should have used the one that came with the SmartiPi case because it was longer and would have been easier.
- Mounted the Raspberry Pi to the display case. the instructions state I could either use the screws to mount the Raspberry Pi or I could use the hinge cover (bottom of picture). The cover has plastic that is sized to touch the Raspberry Pi when closed so I choose that option.
- Screwed the case back to the display. Note that one of the screws will be covered by the cover and any HAT so I tightened this now instead of waiting until the end to tighten everything.
Raspberry Pi camera module with mounted Raspberry Pi and SmartiPi Touch case. Note that the ribbon that comes with the camera will be just long enough to mount to the top or side LEGO studs. A longer camera cable will be useful since the Raspberry Pi with display and case is going to be placed on a table top, which is not necessarily where you want the camera.
Here is the camera attached to the Raspberry Pi. Note that it goes through the slots in the door and the Sense HAT.
Now, attach the door, then mount the Sense HAT and tighten the screws on the Sense HAT. Note that the screws come with rubber spaces that put the HAT the right distance above the door.
Picamera added to Raspberry Pi in SmartiPi Touch Case on the top LEGO pieces. I used a 4X6 plate because it is long enough to reach the back LEGO mount for the top. (mounting to the side could be done with camera mount as is)
Pi camera mounted to LEGO from back
The working camera. Note that I have the whole thing running off a USB battery. I usually have this connected to a keyboard, and use a touch screen to serve as a mouse. It really needs a longer camera connector ribbon. I tried to mount it on a LEGO stand, but the SmartiPi case had to be right next to the stand.
ESP8266 Thing board based environmental monitor
Published:
The goal of this project was to create an automatic temperature and light level monitor for a bioshelter (greenhouse +) run by The Bible Center Oasis Project (Pittsburgh, PA). Because it would be used to grow flora and fauna, it needed ongoing monitoring of temperature and light levels to confirm that it would support the growth of food, and to monitor the effectiveness of measures being explored to control environmental conditions.
| Homewood, Pittsburgh , PA |
| Bible Center Oasis Project bioshelter in Homewood (Pittsburgh), PA |
As an off-grid, solar powered greenhouse, the bioshelter is not connected to any utilities. So any environmental monitoring solution needed to be low powered and use wi-fi for communication (i.e. no phone line). We also preferred solutions that did not force a subscription to a specific data logging service.
| Temperature, humidity and light monitor |
| Sparkfun ESP8266 Thing Dev board based monitor |
I developed a solution based on the Sparkfun ESP8266 Thing development board. This board provided the Arduino microcontroller to control the project and on-board wifi with antenna. The other major components of the board were:
- Sparkfun ESP8266 Thing Dev board
- SparkFun Atmospheric Sensor Breakout - BME280 for temperature, humidity, and barametric pressure sensors
- SparkFun Luminosity Sensor Breakout - TSL2561 for the luminosity sensor
I first soldered headers onto the ESP8266 Thing dev board and the BME280 and TSL2561 breakout boards. These allowed for use of two mini-breadboards to develop the monitor, which also served as a base for the project when deployed.
The Thing dev board and both breakout boards were set up with Inter-integrated Circuit (I2C), protocol, so both sensor boards were connected to the I2C inputs on the Thing dev board.
The Thing dev board was then programmed to transmit the response to a phant server. Specifically, the data.sparkfun.com server that is made freely available by Sparkfun. In addition to being a data display, the phant server allows for accessing the data as csv or json files for further analysis, or using analog.io (link opens up the live graph of data. analog.io an IoT platform made available by Luke Beno). Because phant also exposes the data in JSON format, my usual way of working with the data is to use the jsonlite package within R.
| Breadboard diagram of ESP8266 with BME280 and TSL2561 |
Arduino *.ino code
1: // Include the ESP8266 WiFi library. (Works a lot like the
2: // Arduino WiFi library.)
3: // Uses BME280 and TSL2561 to record temperature, pressure, humidity, and lux data to phant
4: // Code
5: //
6: #include ;
7: #include ;
8: #include ;
9: #include "Wire.h"
10: #include "SPI.h"
11: // Include the SparkFun Phant library.
12: #include ;
13: // Include SparkFun BME280 library
14: #include "SparkFunBME280.h"
15: //Global sensor object
16: BME280 mySensor;
17: // SFE_TSL2561 object
18: SFE_TSL2561 light;
19: //////////////////////
20: // WiFi Definitions //
21: //////////////////////
22: const char WiFiSSID[] = "ssid";
23: const char WiFiPSK[] = "psk";
24: /////////////////////
25: // Pin Definitions //
26: /////////////////////
27: const int LED_PIN = 5; // Thing's onboard, green LED
28: const int ANALOG_PIN = A0; // The only analog pin on the Thing
29: const int DIGITAL_PIN = 12; // Digital pin to be read
30: ////////////////
31: // Phant Keys //
32: ////////////////
33: const char PhantHost[] = "data.sparkfun.com";
34: const char PublicKey[] = "publickey";
35: const char PrivateKey[] = "privatekey";
36: /////////////////
37: // Post Timing //
38: /////////////////
39: const unsigned long postRate = 1000*60 * 30;
40: unsigned long lastPost = 0;
41: // Global variables for TSL2561:
42: boolean gain; // Gain setting, 0 = X1, 1 = X16;
43: unsigned int ms; // Integration ("shutter") time in milliseconds
44: void setup()
45: {
46: // initHardware(); // Setup input/output I/O pins
47: connectWiFi(); // Connect to WiFi
48: digitalWrite(LED_PIN, LOW); // LED on to indicate connect success
49: //For I2C, enable the following and disable the SPI section
50: mySensor.settings.commInterface = I2C_MODE;
51: mySensor.settings.I2CAddress = 0x77;
52: //***Operation settings*****************************//
53: mySensor.settings.runMode = 3; // 3, Normal mode
54: mySensor.settings.tStandby = 0; // 0, 0.5ms
55: mySensor.settings.filter = 0; // 0, filter off
56: //tempOverSample can be:
57: // 0, skipped
58: // 1 through 5, oversampling *1, *2, *4, *8, *16 respectively
59: mySensor.settings.tempOverSample = 1;
60: //pressOverSample can be:
61: // 0, skipped
62: // 1 through 5, oversampling *1, *2, *4, *8, *16 respectively
63: mySensor.settings.pressOverSample = 1;
64: //humidOverSample can be:
65: // 0, skipped
66: // 1 through 5, oversampling *1, *2, *4, *8, *16 respectively
67: mySensor.settings.humidOverSample = 1;
68: // Initialize the SFE_TSL2561 library
69: // You can pass nothing to light.begin() for the default I2C address (0x39),
70: // or use one of the following presets if you have changed
71: // the ADDR jumper on the board:
72: // TSL2561_ADDR_0 address with '0' shorted on board (0x29)
73: // TSL2561_ADDR default address (0x39)
74: // TSL2561_ADDR_1 address with '1' shorted on board (0x49)
75: // For more information see the hookup guide at: https://learn.sparkfun.com/tutorials/getting-started-with-the-tsl2561-luminosity-sensor
76: light.begin();
77: Serial.begin(57600);
78: Serial.print("Program Started\n");
79: // The light sensor has a default integration time of 402ms,
80: // and a default gain of low (1X).
81: // If you would like to change either of these, you can
82: // do so using the setTiming() command.
83: // If gain = false (0), device is set to low gain (1X)
84: // If gain = high (1), device is set to high gain (16X)
85: gain = 0;
86: // If time = 0, integration will be 13.7ms
87: // If time = 1, integration will be 101ms
88: // If time = 2, integration will be 402ms
89: // If time = 3, use manual start / stop to perform your own integration
90: // Use time = 1 so that the midday sun does not lead to an error
91: unsigned char time = 1;
92: // setTiming() will set the third parameter (ms) to the
93: // requested integration time in ms (this will be useful later):
94: Serial.println("Set timing for TSL2561...");
95: light.setTiming(gain,time,ms);
96: // To start taking measurements, power up the sensor:
97: Serial.println("Powerup light sensor...");
98: light.setPowerUp();
99: // The sensor will now gather light during the integration time.
100: // After the specified time, you can retrieve the result from the sensor.
101: // Once a measurement occurs, another integration period will start.
102: Serial.print("Starting BME280... result of .begin(): 0x");
103: delay(10); //Make sure sensor had enough time to turn on. BME280 requires 2ms to start up.
104: //Calling .begin() causes the settings to be loaded
105: Serial.println(mySensor.begin(), HEX);
106: }
107: void loop()
108: {
109: unsigned int delaytime;
110: Serial.println("Posting to Phant!");
111: if (postToPhant())
112: {
113: lastPost = millis();
114: Serial.println("Post Suceeded!");
115: }
116: else // If the Phant post failed
117: {
118: Serial.println("Post failed, will try again.");
119: }
120: delaytime = postRate;
121: delay(delaytime); // Short delay, then next post
122: }
123: void connectWiFi()
124: {
125: byte ledStatus = LOW;
126: Serial.println();
127: Serial.println("Connecting to: " + String(WiFiSSID));
128: // Set WiFi mode to station (as opposed to AP or AP_STA)
129: WiFi.mode(WIFI_STA);
130: // WiFI.begin([ssid], [passkey]) initiates a WiFI connection
131: // to the stated [ssid], using the [passkey] as a WPA, WPA2,
132: // or WEP passphrase.
133: WiFi.begin(WiFiSSID, WiFiPSK);
134: // Use the WiFi.status() function to check if the ESP8266
135: // is connected to a WiFi network.
136: while (WiFi.status() != WL_CONNECTED)
137: {
138: // Blink the LED
139: digitalWrite(LED_PIN, ledStatus); // Write LED high/low
140: ledStatus = (ledStatus == HIGH) ? LOW : HIGH;
141: // Delays allow the ESP8266 to perform critical tasks
142: // defined outside of the sketch. These tasks include
143: // setting up, and maintaining, a WiFi connection.
144: delay(100);
145: // Potentially infinite loops are generally dangerous.
146: // Add delays -- allowing the processor to perform other
147: // tasks -- wherever possible.
148: }
149: Serial.println("WiFi connected");
150: Serial.println("IP address: ");
151: Serial.println(WiFi.localIP());
152: }
153: void initHardware()
154: {
155: Serial.begin(57600);
156: pinMode(DIGITAL_PIN, INPUT_PULLUP); // Setup an input to read
157: pinMode(LED_PIN, OUTPUT); // Set LED as output
158: digitalWrite(LED_PIN, HIGH); // LED off
159: // Don't need to set ANALOG_PIN as input,
160: // that's all it can be.
161: }
162: int postToPhant()
163: {
164: // LED turns on when we enter, it'll go off when we
165: // successfully post.
166: digitalWrite(LED_PIN, LOW);
167: // Retrieve the data from the device:
168: unsigned int data0, data1;
169: double lux; // Resulting lux value
170: boolean good; // True if neither sensor is saturated
171: if (light.getData(data0,data1))
172: {
173: // getData() returned true, communication was successful
174: Serial.print("data0: ");
175: Serial.print(data0);
176: Serial.print(" data1: ");
177: Serial.print(data1);
178: // To calculate lux, pass all your settings and readings
179: // to the getLux() function.
180: // The getLux() function will return 1 if the calculation
181: // was successful, or 0 if one or both of the sensors was
182: // saturated (too much light). If this happens, you can
183: // reduce the integration time and/or gain.
184: // Perform lux calculation:
185: good = light.getLux(gain,ms,data0,data1,lux);
186: // Print out the results:
187: Serial.print(" lux: ");
188: Serial.print(lux);
189: if (good) Serial.println(" (good)"); else Serial.println(" (BAD)");
190: }
191: else
192: {
193: // getData() returned false because of an I2C error, inform the user.
194: byte error = light.getError();
195: printError(error);
196: }
197: // Declare an object from the Phant library - phant
198: Phant phant(PhantHost, PublicKey, PrivateKey);
199: // Add the three field/value pairs defined by our stream:
200: phant.add("temp_f", mySensor.readTempF());
201: phant.add("humidity", mySensor.readFloatHumidity());
202: phant.add("pressure_kpa", mySensor.readFloatPressure()/1000);
203: phant.add("lux", lux);
204: // Now connect to data.sparkfun.com, and post our data:
205: WiFiClient client;
206: const int httpPort = 80;
207: if (!client.connect(PhantHost, httpPort))
208: {
209: // If we fail to connect, return 0.
210: return 0;
211: }
212: // If we successfully connected, print our Phant post:
213: client.print(phant.post());
214: // Read all the lines of the reply from server and print them to Serial
215: while(client.available()){
216: String line = client.readStringUntil('\r');
217: //Serial.print(line); // Trying to avoid using serial
218: }
219: //Print each row in the loop
220: //Start with temperature, as that data is needed for accurate compensation.
221: //Reading the temperature updates the compensators of the other functions
222: //in the background.
223: Serial.print(mySensor.readTempC(), 2);
224: Serial.print(",");
225: Serial.print(mySensor.readTempF(), 3);
226: Serial.print(",");
227: Serial.print(mySensor.readFloatPressure(), 0);
228: Serial.print(",");
229: Serial.print(mySensor.readFloatAltitudeMeters(), 3);
230: Serial.print(",");
231: Serial.print(mySensor.readFloatAltitudeFeet(), 3);
232: Serial.print(",");
233: Serial.print(mySensor.readFloatHumidity(), 0);
234: Serial.print(",");
235: Serial.print(lux);
236: Serial.println();
237: // Before we exit, turn the LED off.
238: digitalWrite(LED_PIN, HIGH);
239: return 1; // Return success
240: }
241: void printError(byte error)
242: // If there's an I2C error, this function will
243: // print out an explanation.
244: {
245: Serial.print("I2C error: ");
246: Serial.print(error,DEC);
247: Serial.print(", ");
248: switch(error)
249: {
250: case 0:
251: Serial.println("success");
252: break;
253: case 1:
254: Serial.println("data too long for transmit buffer");
255: break;
256: case 2:
257: Serial.println("received NACK on address (disconnected?)");
258: break;
259: case 3:
260: Serial.println("received NACK on data");
261: break;
262: case 4:
263: Serial.println("other error");
264: break;
265: default:
266: Serial.println("unknown error");
267: }
268: }
The Honor Was Mine: A Look Inside the Struggles of Military Veterans by Elizabeth Heaney: Book Review
Published:
The Honor Was Mine: A Look Inside the Struggles of Military Veterans by Elizabeth Heaney
My rating: 5 of 5 stars
This is the story of a civilian counselor who joined a program to provide off-the-record counseling services at domestic military bases. She spent short term assignments at each base, with the intention that it was part of a portfolio of provided services, attracting members of the military community who would not be willing to engage the more traditional mental health resources that were already available. It is a view of the military that I very much relate to, as I was attached to a deployed unit as a civilian for a similar period of time.
The first chapter starts slow, she talks about the trauma in her life prior to her taking this assignment as a way of resetting her life (and this gets dropped in the rest of the book). But once you get past the first chapter you get into the story. Two themes that permeate are her being a civilian learning her way around the military environment, which sets up the theme of veterans returning from deployment and those around them responding in their different ways.
The first theme is that of a civilian entering the military community and the differences. She tells the story of her first assignment, of encountering military discipline and curtsy first as a shock, but then respect for the stability that it provides. She goes on to what she realizes the purpose of many of attitudes and rhythms of the military. First, the realization that those in the military must be prepared for anything at any time, and military discipline and curtsy is intended to build the attitudes needed to enable that. Second, that there are a range of overall attitudes held by those in the military, with a considerable more diversity than she expected when she was in a civilian life. But one set of values that holds people together is a focus on duty (towards country and each other, not necessarily in that order) and integrity. And the dissonances between those values and what they see in the civilian world (and in what I called Big Army for that matter) causes many of the problems they have in adapting to home.
But what this long discussion of the difficulties she had in getting used to the military culture does is leads to her discussion about the issues that veterans and their families and the problems that veterans who have returned from a combat zone and their families. And she has gone through the same type of struggle in the first chapter with what she dealt with in her life and the reactions of those around her, and dealing with the same culture shift in the other direction. This part is the deepest, she alternates between telling the stories of the people she meets during the course of her assignments and the story of her processing these. Stories of working with the returning soldiers, the transition between combat and their families, soldiers and spouses trying to deal with the uncertainty of life in general and of soldiers returning after a year of combat, spouses worried about infidelity when the returning soldier is still working through the transition between combat and peacetime, of the care that soldiers have of the wounded and fallen brothers (and sisters) in arms.
Through it all, what shows through is her thankfulness that she has seen this part of life, the pride, dedication, and duty to others that permeates the military. And the recognition that this is not the norm in U.S. society at large. In my own experience, I’ve been told to view the lives of companions as not precious, had offers to refer me to counselors who would tell me that I should have ignored the calls for help from someone in the backcountry. And this contrasting incredibly with being deployed to a combat zone where I was with people who did their duty and tried to do right by those that they were along side. And when I returned, my then girlfriend and I spent a month trying to figure out if I had changed. (and when we were all settled, I figured that I had a good thing going and proposed to her!)
It is a wonderful book. It has a great treatment of PTSD, because it takes the first step of having to experience some of the causes (the need to be constantly aware, the sense that your world and your attitude may have to shift in a heartbeat, and you have to be ready for it, and having to deal with a civilian world that does not generally value duty or integrity) It does not preach, but it gives a set of eyes that had to go through similar experiences and uses that as an analogy into that world.
Note: I received a free electronic copy of this through the Goodreads Giveaways program. The opinions are my own and were not subject to any review.
Lessons in teaching: teaching exploratory data analysis with R
Published:
Last spring, I took over a course labeled as information systems engineering. This is aimed at sophomores in engineering. Historically, this course focused on using the MS Access database. I was asked by the department to take this over after several years of commenting that our engineering seniors have inadequate computer programming skills, as evidenced by the amount of effort they spend on their senior projects doing tasks that would have been much simpler if they tried programming. Last year some of the faculty tried experiments in their classes where they had students code in an assignment (generally they asked for C). In every case this went very badly. So they asked me to take this course and change it so that it covered programming and specifically to use R. (I am effectively the primary data analysis faculty here). In keeping with the course title, I chose to focus the course on data analysis, with one month focusing on databases and how to think about data problems (and giving them time to gradually learn R), the rest on exploratory data analysis. I used as the primary text Data Manipulation with R by Phil Spector, and as supplements GGplot2 by Hadley Whickam and An Introduction to Data Cleaning by Edwin de Jonge and Mark Van Der Roo. I presented the CONVO framework for thinking about data problems based on Thinking about Data by Max Shrum.
As freshmen, they would have has CS0 (the Association for Computing Machinery designation of introduction to computer science for non-computer science/electrical engineering majors) material covered over a two course sequence that also covers mathematics for engineering (primarily linear algebra). The language of instruction is primarily Matlab, but they also cover C and, depending on instructor, Python (there is one module that is sometimes covered by Physics faculty, and they like to use Python). For databases, there is another course on databases taught by an adjunct faculty who used to teach databases for information systems.
For tools I used SQLite (more on why this and not MS Access later), SQLite Manager, R, and R Studio. Prior to the end of the previous semester I sent everyone an email with links to videos introducing them to R and R Studio and encouraged them to introduce them to R through typing out a tutorial (I explained that they would actually learn R over the semester, the typing exercise was to ensure they had seen everything once before we actually needed it in class.).
For assessments, there were weekly labs for computer knowledge, exams mostly covered how to think through data problems. A semester project with two milestones (plus completed project) was the main way to assess how well they developed computer programming competency. Each week, we covered one
We had three datasets that I used as teaching and lab examples throughout the course.
- Titanic survivors
- National Survey of Family Growth
- American Community Survey (U.S. Census, Pittsburgh North PUMA)
Some observations and notes
SQLite vs MS Access. I was surprised to find out that MS Access has a relatively low size limit on databases. It was not able to handle either the National Survey of Family Growth (expected) nor could it handle a single PUMA for the American Community Survey (this was a surprise). That meant we had to use SQLite for the entire course. (my Mac students were happy since this put them on equal footing with the PC students). Next time I will just use SQLite. (and use MS Access only to explain why we are not using MS Access)
Learning R. In a pre-class survey, the entire class indicated complete lack of confidence in programming to fulfil a task (expected). I think that the standard programming language belief that it is always easier to learn a second programming language failed in this case, because I did not realize just how bad their first experience was. While the first month was very intentionally a confidence building exercise, I think that for a portion of the class, they really needed to start from scratch. Next time around, I will spend an entire period doing nothing but walking the class through R.
Data manipulation. This included covering data structures (text, dates, dataframes), regular expressions, plyr, reshape, and missing values imputation. Essentially the Hadleyverse v. 1. One issue here was the wide variety of potential topics. While I think every topic got used by someone in their semester project, some of the student evaluations complained about my teaching topics that were not on the exam. Essentially, for people who are only used to computing on numbers, the entire topic of data manipulation seems to be a heavy cognitive load.
Visualization. I taught qplot, but I think that I should have gone straight to ggplot. I think that either I go the traditional route and build every type of plot as an individual entity, or I present the grammar of graphics approach and build plots. Either way, now that I’ve taught it, I don’t think qplot helps in either, and it is a lot less capable. (every groups final project pretty much had to transition to ggplot)
Projects. I let the students find their own datasets and questions, subject to the fact that they had to write the project purpose using the guidelines we covered in thinking about data. The big division in quality of the projects was the richness of the dataset. Next time, I will be a lot more strict on the dataset, in particular, I had a subjective guideline that they should not consider it practical to look at the whole dataset. In some cases, this still was a very small volume, and it made for a trivial and uninteresting report.
Thinking about data. I used Max Strum Thinking about data framework where for a data project, one should identify the COntext, Need, Vision, Outcome. Every week we read a contemporary news article that included a data component (mostly from the fivethirtyeight.com website) Each discussion opened up with class discussion to summarize the article into this CONVO framework, then a discussion of the analysis in the article itself. This actually worked out pretty well. Each exam had at least one CONVO focused question, and generally they did well (and of the people who did not, there were no surprises based on class participation)
News articles. I had a wide range of news articles that we covered in a weekly discussion, drawn mostly from fivethirtyeight.com, the Upshot column from the New York Times, and the data series from the Washington Post. Each article was assigned at the end of the week, for discussion in the Tuesday morning lecture. Discussion opened up with a summary based on the CONVO framework, then we evaluated the data analysis presented in the article, followed by how we could change it to make it better or to answer a different question. These class periods were fun. My goal was to take 15 minutes for each article, in a few cases we were on a roll so we let it go to 30 minutes. I had good participation. And it showed in the CONVO question on exams, and generally people did well when I asked them to imagine a data analysis based on data presented on a test (this was the last part of a multi part question, where the other parts were about the data presented). One disappointing thing was that when it came time for course evaluations, I was rated poorly with how the class material relates to the everyday world (like all engineering courses do). So I have to figure this one out.
Course evaluations. When course evaluations came in, they were roughly a uniform distribution, which makes them very hard to interpret. In addition, comments that expressed weaknesses were mirrored in the comments that expressed strengths. So that meant that I had terrible averages and a chat with my department chair. Fortunately for me, the generally accepted belief is that the broad diversity in the teaching evaluation is due to pushing the students harder (i.e. making them do programming again) and that this is part of improving the department as a whole. Hopefully when he meets the dean to review the faculty the dean agrees with this assessment as well.
Class projects. About a quarter of the projects (teams of1, 2, or 3) were genuinely impressive. Many projects with 100,000s of records, a few with millions of records, several dimensions, and data analysis that used layered visualizations to explore. Most projects were a little more modest, thousands of data points and reasonable visualizations. Some projects were personal in nature (looking at issues in their home towns), others were fun (several projects revolved around music or sports) A number showed evidence of lack of confidence, shown in very unambitious data sets. The issue with this group is how hard to push. One of the known problems with CS0 or CS1 is that they complete destroy people’s confidence in programming, and a substantial portion of those who take one of their courses completely leave the field, or in the case of engineers, avoid programming at all costs in the future.
Next time around:
- Using a framework like CONVO (Max Strum) works. I am pretty sure everyone at least learned how to think about problems and settings.
- Skip MS Access. I think I probably spent too much time on databases and working with the MS Access interface. Next time, going straight to SQL is probably enough, given that the limits on MS Access means that we cannot do interesting datasets.
- I liked using three datasets the entire course. Actually, some of them used the American Community Survey for their semester projects (after reading in multiple PUMA, e.g. an entire metropolitan area instead of only one PUMA).
- One question that I will have to think about is how much of a do-over of CS0 this course will be. Clearly, as it is most of the class seems to get it the second time around and a good portion are pretty impressive. But there is a pretty large fraction that finished CS0 absolutely convinced that programming is forever beyond them.
Data Manipulation with R by Spector: Book Review
Published:
Data Manipulation with R by Phil Spector
My rating: 4 of 5 stars
The quality that programming language based data analysis environments have that menu driven or batch environments do not is the ability to manipulate data. That means transforming data into usable forms, but it also means cleaning data, manipulating text, transforming data formats, and extracting data from free text. While R falls into this category of data analysis environment, almost all of the available material focuses on the application of statistical methods in R. This fills a much needed niche in how to process data. I still do not regard R as my goto tool for data manipulation, but this book means I am more likely to stay in R than otherwise. I used this as a textbook in a lower division data analysis course and the class went from a group that only half remembers Matlab to being able to process and analyze fairly large datasets. A comment I received was “I looked back on the work done in this project and I cannot believe I actually did that!”
The first part of the book is reading in data and writing out results. It discusses both text (csv, delimited, fixed) and working with relational database. One note is that the database they use is MySQL. This was easily convertible to SQLite, which is what I used in my class because my students are not IT savvy. I also used supplementary material for SQL (which is readily available) Then putting things together into data frames.
Next are a series of data types: datetimes, factors, numbers. For people who have only worked in Excel, these are deal breakers. Even using Excel, these are areas that often go unnoticed by students and lead to problems.
Character manipulation is about working with strings and a gentle introduction to regular expressions. For many of my students, they have never manipulated text programmaticly before, so this chapter was quite successful. For Regular expressions, well it provided a taste of it, enough to solve the lab assignment. I supplemented it with other material, but noone was going to learn regular expressions in 5 pages.
The best part of the book was the sections on aggregating and reshaping data. This is what made what my students were doing with R start to look like magic. Aggregations using the apply family of functions, reshape to convert data into long or wide formats, combining data frames, and an introduction to vectorization. This is not going to make anyone a functional programmer, but these are key idioms and Spector spent a lot of time here.
I am not going to prefer R over Python for working with text and manipulating data, but Data Manipulation with R shows how to do some non-obvious things. The examples are all interesting enough to be useful, and they all work as is. And this goes deep enough into some pretty powerful capabilities that expanded my students understanding of what is possible. While it is becoming dated (an update would have to include dplyr), the approaches it provides put the reader well on their way to being an accomplished R programmer, not just someone who feeds data into functions.
Applied Predictive Modeling By Kuhn and Johnson: Book Review
Published:
Applied Predictive Modeling by Max Kuhn
My rating: 5 of 5 stars
I regard this as a more applied counterpart to more methodology oriented resources like Elements of Statistical Learning. So it applies machine learning methods that are found in readily available R libraries. In addition, the author is also the lead on the caret package in R, which provides a consistent interface between a large number of the common machine learning packages.
- Built around case studies that are woven through the text. For each chapter, the math/stats is developed first, then the computational example is at the end, so that the example can develop data manipulation, application of method, then model evaluation. I like this as it allows for more complex and messy data sets than when using a new, small example for each problem. Also allows for better discussions when illustrating the differences between methods.
- Data manipulation/data processing is given a separate chapter early on. I appreciate the attention given to working with the data (e.g. missing value imputation). There are other resources in data handling, but not in the same place as those that address the statistics methodology.
- Emphasis on model evaluation. There is an early chapter devoted to model evaluation. Then each major section of the book has an early chapter devoted to model evaluation of that class of problem. This is in contrast to many books that are built around types of algorithms, and model evaluation is fit in. Methods and algorithms are relatively easy compared to the thought process of determining what is the right thing to do. It figures that this book will be strong in model evaluation when one of the authors is the lead on the caret package in R.
I used this as a supplement in teaching a data science course that I use a range of different resources because I need to cover working with data, model evaluation, and machine learning methods. The next time I teach this course, I will use only this book because it covers all of these aspects of the field.
Lessons in teaching: Fall 2014 edition - Using the news
Published:
I had started using news articles to illustrate class concepts last year, but this year I started doing so from the very beginning and made it a standing part of the class to do this once a week in both of my classes last semester. Of course, taking 10~30 minutes a week out of class came at a price. In both cases it meant a chapter not being covered. And I pointed this out to the class when time came for course evaluations.
Every week I would identify news articles, magazine articles, or blog posts to discuss in class. (or sometimes it was a historical case). The discussion would always open by evaluating the article through a standard technique. In my decision analysis class it was either generating a value tree or a decision tree. In my simulation class it was generally generating an event diagram (i.e. a white board exercise).
The cases included the international and domestic (U.S.) handling of the Ebola outbreak, health care provider reactions to the rise of independent urgent care clinics, potential expansion of a light manufacturer, pulling a goalie in hockey in the final minutes, flu vaccinations, Corporate restructuring, the construction of super containerships, NASA Commercial Space initiatives, waving a (baseball) runner home, handling of death threats (Gamergate), Space Shuttle Challenger launch decision, automation of warehouses and manufacturing, delivery of disaster response services, and hub and spoke airline operations.
For the discussions themselves, it was a great way of getting students interested. Because of the wide range of topics, international, domestic US, health, industrial, sports, space, different people tended to get into different discussion topics, so across all of the cases, a very large portion of the classes contributed to the conversation at some point.
The other big benefit of the cases was to reinforce the modeling aspect of both classes. For decision models this meant many opportunities to consider the value tree and decision trees, even as we moved to other techniques and topics. For simulation, this was the chance to work on modeling while the course focused on analytical and statistical methods. Even for those who did not take part in the class discussion, there were a few discussions that I think were particularly memorable.
One unexpected benefit was that I had an opportunity to engage in give and take. Once we got past the first few sessions, we were comfortable with me questioning some of the responses, or asking to go a little deeper than the first answer. (without scaring the student into a shell) Students also started asking questions, so we occasionally got a real discussion going.
Sometimes it was fairly light and even entertaining discussion (sports were good for that). And there were times that it went very serious (we looked at the decision not to provide protection from threats of mass murder with plans in Utah with Anita Sarkeesian (meaning it also involved the politics of gender oppression). Ebola was also a big one in September.
So, when compared to the loss of a chapter of coverage, I think that this was a worthwhile direction. The course evaluations commented on how much the students liked using current events as examples of material, and that they felt confident in modeling (which is only one or two chapters in each subject). Compared to a chapter of material that would probably be forgotten, I’ll take the tradeoff.
Thinking with Data by Max Shron: Book Review
Published:
Thinking with Data by Max Shron
My rating: 4 of 5 stars
Thinking with data focuses, not on how to do data analysis, but on the questions that one should be asking. It does so in two ways, first through providing an overall framework to looking at situations, then working through a series of topics using examples to serve as plausible paths of decision making. In a fairly short book, it covers the framework, determining purpose, threats to validity, experimental design, and a few extended examples that illustrates both concepts and deviations. It is a useful quick big picture book that is useful for those whose focus has been on the methods of data analysis or for those who do not have a quantitative background but are faced with data questions and need to be able to work with data analysts.
The first part is probably the most rewarding. Max gives a framework of how to frame a data problem. Context (who is interested in the problem, what are their overall goals and why, what is the goal of the project), Need (the specific need that could be solved through the use of the data model), Vision (an understanding of what the results of data analysis would be like), and Outcome (an understanding of how the data analysis results would be used). The end of this framework would be a story that you can tell
Next is a discussion of how the details of the problem could be fleshed out. The content is probably familiar to anyone who has had to work with stakeholders. The valuable portion here are the vignettes of working through this process on projects. In particular the fact that the vignettes are not projects that necessarily go smoothly, so it does not have the idealized feel that many published vignettes do.
Next is a discussion of presenting the results. The focus here is that the results are not the output of the data analysis, but the use of the data analytics methods to construct and argument. And that argument is going to be presented to people who have backgrounds, prior beliefs, prejudices, and sometimes reasons to argue against your findings.
How to address these disputes is through conducting experiments and testing alternative hypothesis. So a section of the book is on defining causality and designing experiments (interventions) to handle different types of alternative hypotheses.
What makes this useful is the framework and the vignettes. It is good for a quick introduction to this area. As others have noted, it is not tightly organized, so after the first chapter with the framework, it is not useful as a reference, but it helps in focusing how to think.
I teach classes on working with data, and one area that is difficult to get across is the concept that there is a unified whole in the topic, not only a bunch of separated techniques. I plan on using much of what is in this book to help provide that unified whole my classes.
Disclaimer: I received a free electronic version of this book as part of the OReilly Bloggers program.
Using blogs and news articles as class mini-cases - How discrete-event simulation can help project prison populations
Published:
How discrete-event simulation can help project prison populations (SAS Subconscious Musings)
My experiment this semester is more intensive use of news articles as subjects for in-class discussion of examples of applications of what we are learning. While I have done this in the past, this semester I made it a deliberate plan to discuss one article a week in each class. So far this semester in my simulation and decision models classes, I have covered reactions to the W. Africa Ebola outbreak, Game 7 of the 2014 World Series, Gamergate, flu vaccinations, commercial manned space transport, pulling a goalie in hockey, cargo shipping, business expansion, business divestiture, automation of manufacturing, health care system operations, among other things.
I identify articles through the use of RSS feed aggregators. My news feed includes a number of feeds from a range of business school professors focusing on supply chain and operations management issues. I follow the CDC MMWR as well as the journal Health Affairs to get health care related articles. And the New York Times front page and Google News are good for a lot of different stories. The key is finding an article where the reporter was good enough to discuss the various options that were available and enough details that you can figure out the values of various actors involved.
The key a good class case article is that there are potentially reasonable alternatives to discuss. In the decision models course, the discussion revolves around identifying the courses of action available, the sequence in which decisions need to be made and information becomes available, assessing the attributes (values) of the people involved, then assessing how they may assess probabilities of various events.
For the simulation course, the focus on case discussions is on understanding how a decision maker in the article may use the simulation, then we do a whiteboard exercise where we draw out an event graph diagram to model that system, focusing on what needs to be included (states, events) based on the decision maker needs. The goal is to discuss modeling in a specific context, so we can talk about what needs to be included, and what does NOT need to be included in the model to fit the particular purpose. The contrast is to the textbook homework problems, which generally provide a very specific context and set of details which have be included in the model to answer the homework problems. Textbook problems generally do not include thinking about modeling in such a way to determine what is the right question and how to simplify the model to address the question.
Last week we looked at the decisions made by the North Carolina Sentencing commission. Unlike most cases, in this case we happen to know for a fact that a simulation was used in the decision making process.
Our discussion began with purpose: why would the North Carolina Sentencing Commission be interested in a simulation of prison population. We came up with the need to plan prison space, make arrangements with neighboring states to house NC prisoners, and to allocate resources to monitor parolees.
Next, a discussion of what would the simulation need to track to fulfill the purpose of the NCSC. This would include the number of prisoners and the number of parolees. And the time remaining for each prisoners sentences. Then, we have the state of the system being the prisoners and parolees, and the terms of sentencing. (we decided not to discuss the size of the prison, since that is something that was being determined).
The last part of the discussion was where the typical homework or exam problem started, diagramming the events tracked by the system, how each events changes the system state, and then how to generate delays in the simulation.
The purpose of the exercise was to discuss modeling. Not in terms of how you build a model from a system description, but to think through how to model and make modeling trade-offs given the decision that needs to be made about a specific system. The cost of this discussion is time, doing this results in us not completing a semester syllabus of a class that is quite analytical. But, as textbooks usually begin modeling examples with a system description and a purpose, I think it adds to the course and I think the compromise is worth it.
Thanks to Natalia, Jeff, and Leo from SAS for our conversations about this particular SAS case at the INFORMS conference. It enriched the class discussion to know what was happening behind the scenes of the Subconscious musings blog article.
Testing in pairs: The Global Day of Coderetreat
Published:
Global day of Coderetreat: a day to celebrate passion and software craftsmanship
A Coderetreat is like a master class for computer programmers. It is a chance to view programming as a craft that can be practiced and honed. And like other crafts, the way you develop skill and creativity is to create limits, then use your creativity to accomplish the goal while working around the limits.
As an engineer, I am not primarily a programmer, although I have some level of skill, so this is not the typical view of programming, which is viewed more as a tool and a necessary evil. The effect is that improved competency is not valued, which leads to inability to deal with dirty data, models that cannot be implemented, and results that cannot be reproduced.
Code retreat is build around the four principles of simple design, due to Kent Beck:
- Runs all the tests
- Expresses every idea that we need to express
- Says everything once and only once
- Has no superfluous parts
The structure of the Coderetreat is six sessions where we work with Conway’s Game of Life. For each session, we pair with a different partner. In addition, in each session we are to begin from scratch, and there is a twist to the rules. The goal was never to actually implement the Game of Life (although in two cases we actually had all of the parts working and tested), but to spend time working with someone else on code.
Observations
Pair programming. This was my favorite aspect of the Coderetreat, pairing with six different people. I figure there were two where I was generally more skilled, two where I was generally less skilled, and two where we were pretty much even. In every case our end solutions had very different designs as it was a combination of our different ways of looking at things and our experiences of having tried different designs in previous sessions, and the skill levels of the people involved. When I worked with students, we would occasionally have a session where we worked together to solve a problem, and some of my students have commented that they found those sessions to be invaluable because they had a chance to watch how I worked and saw how I dealt with different types of problems. But this time I did pair programming on people on a much more even footing and I get to experience it as well. It showed in how we used different tools (although I was experimenting with a new IDE), how we solved problems in code and how we solved logic problems.
Test driven development (TDD). I’ve heard of the concept before, and I have even contributed to a unit test framework, but I’ve never really done it. What TDD did was to encourage more modular code. It also forced us to put more thought into our design, as we had to consider what information was required an in what format to do what we needed. In one session, one member of the pair would write tests and the other would write the code, and the two were not allowed to communicate. As the one writing the tests, since we could not otherwise communicate, I realized that in writing the tests I was forcing a set of data structures and a design in my tests.
Throwing away dsigns. We started each session with a clean code base. What it meant was that we did each session using the lessons from what went before. The first two sessions we did not get much progress, but the third was the one where we made the most progress, as we basically learned from the combined mistakes made over the first two sessions and designed the tests with the past problems faced in mind, and the solution was fairly easy after that. That was good because the next three sessions were the ones with the wierder twists.
New languages. Python is by far my strongest language, but I did one session in Clojure and one session with Java. In both cases I learned a lot about how people set up their tools and the idioms they used, which were different than what you see in standard texts.
Dealing with constraints. There were three weird twists. One was mute pairs, one was limitations on the size of methods, one was no use of conditional statements. Mute pairs forced the design to be simple and clear (especially difficult because we did it in Clojure, which I barely can say I know without the aid of a book in front of me), the size of methods led us to generate very ugly method/class hierarchy to deal with the extreme restrictions, no conditionals lead to a range of creative hacks. This has an effect similar to a lot of exercises done in the creative arts, adding constraints is one way of encouraging more creativity.
This was a valuable experience. Most of the people there had computer science backgrounds, and pairing with them taught me a lot. And I was somewhat glad to know I could add to people’s knowledge base as well.
Thanks to Code & Supply (@codeandsupply) and Think Through Math (@ThinkThroughMath) for making this event possible and for helping to keep it free, and to IBM for hosting.
Flask Web Development by Grinberg: Book Review
Published:
Flask Web Development: Developing Web Applications with Python by Miguel Grinberg
My rating: 4 of 5 stars
I’m not a web developer, but Flask has always had an appeal of potentially being a potential front end to a database based application. But I’ve never gotten very far as tutorials generally look at only the main application, and I’m pretty sure I need some extensions but I have not been able to assess the quality of the many plugins available. Flask Web Development is that tutorial that shows Flask and selects quality extensions to introduce.
Flask Web Development is written as a tutorial, not a reference. As such Grinberg can decide on what is important. It starts like many other Flask tutorials in looking at the basic application structure, templates and web forms. But then it looks at databases along with a set of extensions for database management built around SQLAlchemy: Flask-SQLAlchemy, Flask-migrate. (and email, which I don’t do)
He brings it all together with walking you through a blogging application. What gives the tutorial an over-the-shoulder feel to it is an innovative use of the github repository that goes with the book. Instead of having source files in the repository, the repository uses tags to incrementally build source files, so it is really like working alongside someone who knows what they are doing as they build the application. One tag will have a basic working version of functionality, and checking out subsequent tags builds out more features.
Flask Web Development covers many aspects of web programming, well beyond what most Flask tutorials will cover. I appreciate the deep dives into database management with SQLAlchemy, and the sections on testing and profiling which have applications beyond web development.
Disclaimer: I received a free electronic copy of Flask Web Development as part of the Oreilly Blogger programming.
[
]
Clojure for Machine Learning by Wali: Book review
Published:
Clojure for Machine Learning by Akhil Wali
My rating: 3 of 5 stars
For a book that is about [programming language] for [computational task], there are two approaches. One is to assume basic competence in the language and teach the task, the other is to assume that the reader has basic competence in the task and teach how to do it in a language. This book assumes knowledge of Clojure and tries to teach machine learning. But I find that it does just enough to be dangerous. It has a series of methods but does not provide discussion on why you would choose once class of method over another, and it completely skips model evaluation. What this creates is someone who has a good idea of the mathematics and implementation of methods, but not when to use it or if it actually did what was intended.
This makes me slightly different than the ideal audience of this book. I am learning Clojure and have only started using it for data analysis in real life. However, I have used Python and R for data analysis for several years now, and I have use both of them (and trained students to help me using both) for different machine learning projects (and I use R for teaching a course in data science).
Each chapter in Clojure for Machine Learning takes a look of a class of machine learning algorithms and takes several looks at it. Generally, Wali looks at the mathematics and theory of the algorithm, then a simple implementation in Clojure, then some examples of using existing library implementations on a problem. The mathematical treatment seems nice, but it would not compare to an actual text on machine learning/data mining. And while seeing an implementation in Clojure has some value, I would have liked to have seen more humility in doing so (i.e. some recognition that there are limitations of an implementation that can actually fit into a book of this size with everything else that needs to be done).
Two things that bother me about this after I finished was realizing that for each class of algorithms, the book only covers one or two methods. Which is fine, but it does not even acknowledge that there is a greater world. And as there is no discussion on how to perform model evaluation, an enthusiastic reader may reach the conclusion that they know what they are doing when implementing them against a data set and problem. Essentially, the enthusiastic reader knows enough to be dangerous and does not know what he does not know. If I were to suggest this book to someone, it would have to go with a severe caveat that what you know after this is how to set up a machine learning problem. More research has to be done to determine what actually needs to be done (the libraries used are much broader than what is covered) and then, learn from somewhere else how to evaluate or tune the methods used.
In the end, I would treat this as a book of examples or cases of Clojure being used in machine learning. There is room in the world for a book on Clojure for machine learning, but this is not it.
Clojure Data Analysis Cookbook by Eric Rochester
Published:
Clojure Data Analysis Cookbook by Eric Rochester
My rating: 5 of 5 stars
This book is a good tutorial on data science using Clojure. It starts with working with data (access and cleaning up), then the various chapters cover a range of tasks from focusing on numeric computing (performance, parallel processing), statistics (Incanter and other numerical libraries), working with other numerical software (Mathematica and R), graphics, and the web. The topics are actually a fairly complete look at data science, so it feels more like a tutorial than a cookbook.
I found this to be a great text on working with data in Clojure. My background is in technical computing, mostly R and Python although I also use C, Java, and Fortran as needed. I’ve been dabbling in Clojure, but I had not made the jump from doing tutorials and exercises in Clojure to doing something for real. Working through this book has improved my skills in setting Clojure up and using it for real tasks. The book code also provides a nice example of good programming style (I think) that I can see myself trying to emulate.
A book on data science is necessarily about the practical details of implementation, not about mathematical and statistical methods. Presumably, the reader has another source about the details of various statistical and machine learning methods that they can use to figure out what to do, then the Clojure and Incanter API documents tell you how to do it, and this book is about how to do the 80% of data science that is not about implementing the algorithm, but about how to manage the data, then work to communicate the results of the algorithm in understandable ways. This book is probably what brings me from dabbling in Clojure to being able to use Clojure for real tasks. Well done.
Functional Thinking by Neal Ford: Book review
Published:
Functional Thinking by Neal Ford
My rating: 4 of 5 stars
Functional programming can often sound like magic, in its promises of greatly improved productivity and near guarantees of accuracy and expressiveness. But one problem is that the examples usually given are those in an unfamiliar form, using languages very different than the procedural and object oriented languages most of us are used to. And the advantages are not apparent when presented. What Ford does in Functional thinking is to present the advantages of functional programming within the context of an object oriented language that can support this, then move to more appropriate languages to show how the advantages can be more clear when the language supports it more directly. It does not stand alone, you will not learn functional programming from this book, but it does offer a more clear argument for why functional programming can useful and better in some circumstances.
The setting that I have seen functional programming explained and taught has always been in the context of demonstrating a functional programming language, such as a Lisp (Lisp, Scheme, Clojure). Or sometimes, a language that has some functional features (R, Python). But I always had to take the commentators word at why this was good. Ford using Java 8 (as it adds some functional features), Groovy, Scala, and Clojure provides a progression from functional features in an object-oriented language (Java), to functional features in multi-paradigm language (Groovy and Scala), to it look in a language that is clearly functional (Clojure). This provides a good look at its qualities by showing how these features improves upon an object-oriented solution, then how it is more expressive and closer to the problem when presented in the cleaner form (i.e. in Groovy, then Clojure). I probably would not have caught the lessons of this if this was my first exposure, and I stopped after the first chapter to watch one of Ford’s talks on YouTube so I could get an overview of the book before I finished. But the reward is that it gives context for my learning of Clojure (and functional programming in general).
Not for learning a programming language, but for learning how to think about problems in new ways, enabled by functional programming.
Note: I received a free electronic copy of this for review as part of the O’Reilly Press Blogger program.
Notes from teaching data science for the first time
Published:
| Drew Conway Data Science Venn Diagam |
I spent this past semester teaching a course in data science. While there has been a data mining course taught in the department, it is offered irregularly and had a different focus. The premise for the course I taught was that data science was the intersection of data hacking, mathematical and statistical methods, and domain knowledge (with props to Drew Conway). The students I had generally had little to no programming experience (or meaningless background). All have had a first course in statistics.
I used two texts. First was Stanton Introduction to Data Science, which is used in the Syracuse Data Science certificate program. Second was Introduction to Data Mining with R by Luis Turgo. All of the students were also told to go through Introduction to R prior to the beginning of the course (or as early as possible).
The class started off going through Introduction to Data Science, which included a few introductory chapters to data analysis, and introduction to R and the R Studio IDE. Then were chapters on some basic methods at the basic level such as text processing, review of regression. Then additional methods such as association rules and support vector machines. We then switched to Data Mining with R which were a series of case studies. Each case study had some form of data munging (manipulation) required, with the first one having an involved demonstration of how to handle missing values, either determining the correct value or removing as appropriate. Each case also had a lengthy discussion of the methodologies used, with what each is being used for and a basic understanding of how it worked and its implementation using libraries in use with R (there is a book package, but it has mostly data sets and some functions to assist in data manipulation and visualization.
The assignments were built around individual projects. Their were three presentations, exploratory data analysis, preliminary data analysis, then final. The first two they could work together, but the final one had to be solo as they needed to have individual topics (even if they used the same data sets). The intent was that these assignments would build towards a final goal (but they had flexibility to bail if they wanted to mid-semester.
Probably 1/4 of the students found projects off of Kaggle, which is useful because it has a nice complex data set and comes with a legitimate question. Another 1/4 of the students used public health as a motivating area (University of Pittsburgh is home to Project Tycho, which is a rich dataset of infectious disease in the U.S., also, there is a joint program with the Department of Industrial Engineering and the School of Public Health).
Some problems that came up. First, I discovered that many of the students had an operating assumption that all data was normally distributed, and they constantly made claims that their data was normal. Even when the data was noticeably skewed. This was embarrassing when they would make statistical tests, and the test graphic would include the corresponding normal approximation which was nowhere near the data. I eventually figured out that for many of them, when they took statistics they were constantly fitting normal distributions in their homework data sets, so I explained that their textbook problems were written so that their would be a normal distribution to find.
Another problem was the lack of a hypothesis. Many students started to pick problems that could be solved through linear regression and declared that because it met a p-value criteria they were done. (and in some cases, I recognized the data set as being a teaching data set). But even though they could fit a regression, there was no theory on why the data related in a given way. Essentially, they were pushing data through an algorithm without any subject understanding. Most (not all) of them got the ideas by the end of the second presentation.
A third difficulty was skipping the model evaluation. Most of the methods covered have some parameter that was the analysts choice, so they should have explained how they chose the value of that parameter. Generally, this should have been a discussion of making the tradeoff between closely approximating the observed data and overfitting. Some students skipped this completely (essentially, this is what would happen if you fed data to an algorithm then reported the result using all default values)
One big observation I had by the first presentation was being able to identify the level of programming ability by the choice of projects. I strongly suspect that a number of students were minimizing the programming required. But that became reflected in the level of ambition of the projects. Non-programmers tended to choose simplistic data sets with little variety. I think the difference is the workload. People who could program were able to slice the data available on a multitude of dimensions without regard to scale, since the computer would do all the repetitious work, while those who could not program generally were reluctant to have large sample populations or multiple data sets on the same population.
Things for next time. First, impress on them the need to learn to program. Essentially, the projects from those who could program were so much richer than those who could not (even at a low level of programming skill) that I was embarrassed for those who could not program. Second, I should push harder on the need to have a hypothesis that was driven by domain understanding of the problem. This should be pushed harder from the very beginning to discourage people from merely pushing data through statistical methods and reporting results.
For teaching data mining, I think that the organization of the course needs more methods focus. The principle text was case driven, but that meant that methods were being introduced in a fairly arbitrary sequence. I ended up doing a methodology focused review over the last few weeks. What I should do next times is after the introductory section (Stanton Introduction to Data Science), have the next several lectures be a tour of the classes of data mining methods (regression, classification, clustering, feature selection), then do the case studies. One resource I found useful in this are articles from the Journal of Statistical Software, many of which are focused on R packages that implement classes of methods.
This was a very good course. I wished that the students did more participation (by the final presentation, there were some points that were given based on shear quantity of comments, which several students took advantage of). Some of the projects were much more ambitious than any other done in the MS program. And I have a lot stronger argument about the need for the graduate students to know scientific programming as a skill set.
Agile Data Science by Russell Jurney: Book review
Published:
Agile Data Science: Building Data Analytics Applications with Hadoop by Russell Jurney
My rating: 4 of 5 stars
One of the problems with data science is that any description of what is encountered takes on the appearance of a mythical unicorn, noone person could possibly have all of the skills required. And it gets worse when you add to the standard set of statistics, domain knowledge, and programming the ability to deploy the application into a high speed environment. This book is not going to make a data scientist an expert in running a data center, but it is useful to give someone who has the rest of the skills an understanding of the environment their work will be deployed into.
One of the conflicts between the data scientist/analyst and information technology groups is that while the data scientist gives the data owned by the organization its value, IT is charged with storing the data and providing the access. And in a high velocity, high volume environment of big data, not understanding how the architecture works can lead to the data scientist creating valid solutions that cannot be applied in the actual day to day working environment. That is where this book comes in. The book has associated virtual machines in software repository so that the data scientist who does not know anything about infrastructure and the software stack that the data and the analysis rides on can see how everything fits together.
The book title is misleading. This is not a book about data analytics. This is a book for data analysts so they know how their analytical application is deployed and applied to day-to-day use in enterprise environments. For that reason it is useful.
Disclaimer: I received a free electronic copy of Agile Data Science as part of the Oreilly Press Blogger program.
Doing Data Science by O’Neil and Schutt: Book Review
Published:
Doing Data Science: Straight Talk from the Frontline by Rachel Schutt
My rating: 4 of 5 stars
Doing Data Science is about the practice of data science, not its implementation. It is based on a course on data science that featured a guest lecturer on each topic. This leads to the guest lecturers (and chapters) focusing more on important concepts rather then the methodology. So, this is not a textbook or a how-to-do-this type of book, rather it is a how-to-think-when-doing book.
A problem with books like this where each chapter is written by someone different is the need for coherence. A second is that each author typically has something to day, and she has to say it in her chapter. So, compared to other data science books, it suffers from the chapters not building on each other in a systematic way and having multiple messages that appear as you go through the book.
One benefit from this is that each author has something to say. While I find the book thin on how to do things, this is a good source of wisdom in why things are done and issues that come up along the way in real life. I am teaching data science for the first time and I find myself turning here for topics of discussion which my chosen textbooks don’t cover (as they have more focus on how to do things).
I don’t think this is the book to use to learn how to do data science, and I suspect the students at Columbia learned how to find other sources to help them figure things out. But it provides wisdom, which is harder to find and worth quite a bit.
Note: I received a free electronic copy of this book from the publisher as par of the OReilly Bloggers program.
Data Points by Nathan Yue: Book review
Published:
Data Points: Visualization That Means Something by Nathan Yau
My rating: 4 of 5 stars
This book is not about how to create data visualizations, it is about how you use visualizations to communicate data. In that respect it is not trying to be a book about tools, but a book on aesthetics, it focuses on how you evaluate different combinations of visualization options for communicating different types of information about data, not just a number of rules. In this respect, it goes considerably deeper and profound about how people comprehend and interpret visualizations than a set of pithy rules masquerading as common sense. In this respect, it is a successor to Tufte in an age where being able to try alternative visualizations and even having consumers interact with the visualizations is cheap.
The book is not a description of various types of visualizations, even though it has such descriptions and discussion of comparative assessments. It is a book on how to think about the message(s) you are trying to communicate, and how to do so in ways that can engage the reader at many layers of depth where simple messages are easily grasped, and complex messages can be absorbed with their relations and implications. Along the way he discusses the relative strengths of using different types of visual cues to communicate information (position, length, angle, direction, shapes, area, volume, saturation, hue), which is much deeper than saying ‘bar charts are better than pie charts’ (which is an argument that a post-doc tried to engage in with me once). After a brief introduction, he proceeds to show you by example after example of the comparative qualities of each cue, and also how they can be used in combination to show multiple levels of information and relationships.
One of my biggest insights from ‘Data Points’ is actually not discussed in the book. The book gives you the understanding you need to evaluate the range of combinations of means of presenting data. But about halfway through I realized that the discussion and philosophy of combining these visualizations has a name. Wilkinson’s Grammer of Graphics. I have learned the ggplot implementation of Grammer of Graphics, and I favor it above other plotting families in R and Python as being more flexible and giving you more control over the result. The discussion in Data Points explains why Grammer of Graphics is important, it provides an interface for exploring combinations of aesthetics (visual cues) to communicate aspects of complex data sets. And with this, it will probably change how I present and teach visualization for data analysis.
Data Science for Business by Provost and Fawcett: Book review
Published:
Data Science for Business: What you need to know about data mining and data-analytic thinking by Foster Provost
My rating: 5 of 5 stars
What Provost and Fawcett have done is to write a book on data mining that focuses on the why of data mining technique, which is great complement to all the books that focus on the how of data mining. And because it focuses on the why for the myriad of methods that fall under the heading of data mining, this would be a good source for a manager of a project for which data mining was merely part of the project, or for a source of good explanations when you need to explain to others what data mining methods (or buzzwards) can and cannot do.
I’ve come across a number of data mining books. Some are deep into the mathematics and statistics that underlie the methods of data mining. Others focus on how you implement methods. But while this helps with technique, a missing niche is the why, or the morality of data mining methods. They go over a range of methods, but the focus is on the task, recognizing what kinds of questions can be asked in a situation, then how to answer it. This is different from a methods book that has chapters focused on PCA, SVM, trees and forests, or other techniques. The second can lead to tossing out buzzwords. This book is the first, and is for having conversations about how to get a task done.
While I’ve read and worked through examples from books that focused on methods and implementations, I think that my understanding of data mining has improved significantly on reading this book. I’m recommending it to a former student who has since had to learn and implement these methods in practice, so he can better explain what he has done and its significance at his company. My only nit to pick is the title. The book clearly focuses on data mining, not on other aspects of data science. Within that realm, I recommend it unreservedly.
Disclaimer: I received a free electronic copy of this book through the OReilly Blogger program.
portfolio
Get Well Sooner: Pitt Engineers Collaborate with Children’s Hospital of Pittsburgh of UPMC to Reduce ER Wait Times
Published:
Collaborative systems engineering effort analyzing patient flow and operational bottlenecks to reduce Pediatric Emergency Department wait times.
Southwestern Pennsylvania Manufacturers Get Student Support
Published:
Multi-company partnership with Catalyst Connection linking Pitt IE capstone teams to regional manufacturers for operational optimization.
Pitt Industrial Engineering Students Apply Their Knowledge in Collaboration with Grane Rx
Published:
Award-winning capstone project developing production scheduling and cost-optimization models for pharmacy distribution expansion.
INFORMS Pro Bono Analytics Project Wins Volunteer Award from Houston Methodist Hospital
Published:
Recognized with the 2019 Outstanding Community Partner Volunteer Award for volunteer retention modeling and visualization.
IE Students Help a Local Caterer Develop a Recipe for Success
Published:
Pitt Industrial Engineering capstone team designs mechanical production tooling for a local Hill District catering business.
COVID-19 Planning: SIR Models and Regional Healthcare Strategy at Highmark Health
Published:
Technical talk on solving two-population SIR compartmental models to estimate infection peaks and manage regional healthcare capacity.
Students Develop Tool to Help American Red Cross Estimate Shelter Needs After Earthquakes
Published:
A collaboration with Carnegie Mellon University Heinz College students to develop a rapid earthquake shelter demand forecasting model.
Skills That Transfer: How Red Cross Service Powers Data Scientist’s Career
Published:
Feature profile highlighting over 20 years of disaster planning service and how volunteer fieldwork informs enterprise data science.
What’s Your StORy: Changing Healthcare, Changing Lives – With Data Science
Published:
Video interview on operations research, healthcare simulation, the technical translation gap, and Red Cross disaster planning.
RealLIST Innovators 2025: Meet 15 People in Pittsburgh Pushing Science and Tech Forward
Published:
Selected as one of 15 Pittsburgh innovators advancing science and technology in 2025.
Applying generative AI in analytics: failure modes and opportunities
Published:
Webinar demonstrating how to integrate Generative AI into data science workflows while navigating technical fault lines and common failure modes.
publications
Modeling Emergency Medical Response to a Mass Casualty Incident Using Agent Based Simulation
Published in Socio-Economic Planning Sciences, 2012
In this work, we built an agent based model of a given urban area to simulate the emergency medical response to a mass casualty incident (MCI) in that area.
Recommended citation: Wang, Y., Luangkesorn, L., & Shuman, L. (2012). 'Modeling Emergency Medical Response to a Mass Casualty Incident using Agent Based Simulation.' Socio-Economic Planning Sciences, 46(4), pp. 281–290.
Download Paper
Practice Summaries: Designing Disease Prevention and Screening Centers in Abu Dhabi
Published in Interfaces, 2012
This article outlines the operational design and implementation of regional disease prevention and screening facilities in Abu Dhabi
Recommended citation: Luangkesorn, K. L., Norman, B. A., Zhuang, Y., Falbo, M., & Sysko, J. (2012). 'Practice Summaries: Designing Disease Prevention and Screening Centers in Abu Dhabi.' Interfaces, 42(4), 406–409.
Download Paper
A Sequential Experimental Design Method to Evaluate a Combination of School Closure and Vaccination Policies to Control an H1N1-Like Pandemic
Published in Journal of Public Health Management and Practice, 2013
We used an open-source agent-based modeling system, FRED (A Framework for Reconstructing Epidemiological Dynamic), to simulate the spread of an H1N1 epidemic in Alleghany County, Pennsylvania applying best subset selection procedures.
Recommended citation: Luangkesorn, K. L., Ghiasabadi, F., & Chhatwal, J. (2013). 'A Sequential Experimental Design Method to Evaluate a Combination of School Closure and Vaccination Policies to Control an H1N1-Like Pandemic.' Journal of Public Health Management and Practice, 19(Suppl 5), pp. S37–S41.
Download Paper
Markov Chain Monte Carlo Methods for Estimating Surgery Duration
Published in Journal of Statistical Computation and Simulation, 2015
We combine expert judgement, expert classification of procedures by complexity category and historical data in a Markov Chain Monte Carlo model and test it against one year of actual surgery cases at a multi-speciality surgical suite.
Recommended citation: Luangkesorn, K. L., & Eren-Doğu, Z. (2015). 'Markov Chain Monte Carlo methods for estimating surgery duration.' Journal of Statistical Computation and Simulation, 86(2).
Download Paper
Analysis of production systems with potential for severe disruptions
Published in International Journal of Production Economics, 2016
The produce-to-stock with production disruptions model is applicable to systems where the decision on production rate is coupled with the setting of the base stock level when production disruptions are possible.
Recommended citation: 'Systems with potential for severe disruptions.' (2016). International Journal of Production Economics, 171, pp. 478–486.
Download Paper
Machine Learning of Fire Hazard Model Simulations for use in Probabilistic Safety Assessments at Nuclear Power Plants
Published in Reliability Engineering & System Safety, 2019
This paper details the application of machine learning algorithms to evaluate fire hazard model simulations, streamlining probabilistic safety assessment protocols at nuclear power generation facilities.
Recommended citation: Worrell, C., Luangkesorn, K. L., Haight, J., & Congedo, T. (2019). 'Machine Learning of Fire Hazard Model Simulations for use in Probabilistic Safety Assessments at Nuclear Power Plants.' Reliability Engineering & System Safety, 183, 128-142.
Download Paper
Life Cycle Assessment for Long-Term Production Operation
Published in DAAAM International Scientific Book 2021, 2021
In this study, associative digital technologies, machine learning, and simulation are used to process both operational and related economic data from an actual poultry farm.
Recommended citation: Gaku, R., Luangkesorn, L., & Takakuwa, S. (2021). 'Life Cycle Assessment for Long-Term Production Operation.' DAAAM International Scientific Book 2021, Chapter 11, pp. 131–138.
Download Paper
Data Science and Analytics in Healthcare
Published in Health Informatics: An Interprofessional Approach, 3rd ed., 2024
This book chapter provides an overview of foundational data science applications, methodologies, and analytical approaches utilized within modern healthcare systems.
Recommended citation: Luangkesorn, L. (2024). 'Data Science and Analytics in Healthcare.' In L. R. Hardy (Ed.), Health Informatics: An Interprofessional Approach (3rd ed.). Elsevier. ISBN: 978-0-323-71196-8.
Improving Response Through Advance Operational Planning and Academic Collaboration
Published in OR/MS Today, 2026
Discussion of the development of American Red Cross’s doctrine, procedures, and predictive tools for Advance Operational Planning with emphasis on partnerships with university researchers.
talks
Natural Language Toolkit and Association Rules
Published:
This talk on the Natural Language Toolkit and Association Rules uses the Mine Safety and Health Administration Accident Injuries Data Set as example data was given at the Pittsburgh Python Meetup on October 22, 2014.
Recommended citation: Luangkesorn, L. (2014). 'Natural Language Toolkit and Assocition Rules', Pittsburgh, PA.
Bayesian Methods in Python
Published:
This talk on Bayesian Methods in Python given at the Pittsburgh Python Meetup on May 27, 2015. It gives examples of Bayesian Markov Chain Monte Carlo (MCMC) methods using the emcee, PyMC, and PyStan.
Recommended citation: Luangkesorn, L. (2015). 'Bayesian Methods in Python', Pittsburgh Python, Pittsburgh, PA.
INFORMS 2019 Pro Bono Analytics – Providing Analytics Support to Nonprofit Organizations
Published:
Panel on leveraging analytics expertise for social good and supporting nonprofit organizations.
Recommended citation: Luangkesorn, L. (2019). 'Pro Bono Analytics – Providing Analytics Support to Nonprofit Organizations.' INFORMS 2019 Annual Conference.
INFORMS 2021 Evaluating Specialist Staffing for Telestroke Consult Support for Regional Hospital Emergency Departments
Published:
Operations research modeling of telestroke staffing levels to support emergency department consults.
Recommended citation: Luangkesorn, K. L., & Hackett, C. (2021). 'Evaluating Specialist Staffing for Telestroke Consult Support for Regional Hospital Emergency Departments.' INFORMS Annual Meeting.
INFORMS 2022 Solving the Two Population SIR Model to Provide Early Estimates of Peak and Duration of a COVID-19 Wave
Published:
Conference presentation at INFORMS 2022 on epidemic trajectory forecasting using two-population SIR modeling.
Recommended citation: Luangkesorn, L. (2022). 'Solving the Two Population Sir Model to Provide Early Estimates of Peak and Duration of a Covid-19 Wave.' INFORMS Annual Meeting, Indianapolis, IN.
Tech Talk: Solving the Two Population SIR Model to Project COVID-19 Wave
Published:
Invited technical seminar on epidemic modeling using two-population SIR formulations for ChristianaCare iREACH.
Recommended citation: Luangkesorn, L. (2023). 'Tech Talk: Solving the two population SIR model to project COVID-19 wave.' ChristianaCare iREACH / Delaware-CTR.
Download Paper
The Development of Advanced Operational Planning for National Disaster Response at the American Red Cross (University of Pittsburgh)
Published:
Guest lecture on data for public good and Red Cross operational modeling at the University of Pittsburgh.
Recommended citation: Luangkesorn, L. (2023). 'The development of Advanced Operational Planning for national disaster response at the American Red Cross.' Guest lecture in IE 1171 Data for the Public Good (Prof. Rahimian), University of Pittsburgh.
INFORMS 2023 Panel Successfully Entering the Data Profession
Published:
Panel session discussing strategies and pathways for entering the data science and analytics profession.
Recommended citation: Luangkesorn, L. (2023). 'Successfully entering the data profession.' INFORMS 2023 Annual Conference.
Information During a Disaster: The Development of Advanced Operational Planning Tools and Model(CMU)
Published:
Guest lecture on disaster analytics and decision support tools at Carnegie Mellon University Heinz College.
Recommended citation: Luangkesorn, L. (2024). 'Information during a disaster: The development of advanced operational planning tools and models.' Guest lecture in 94-465 Data Analytics for Decision Making (Prof. Barrios), Carnegie Mellon University.
INFORMS Analytics 2024 Rising Analytics Career Panel
Published:
Career development panel at the INFORMS Analytics Conference.
Recommended citation: Luangkesorn, L. (2024). 'Rising Analytics Career Panel.' INFORMS Analytics 2024, Los Angeles, CA.
POMS 2024 Analyzing Social Vulnerability as a Proxy for Damage in a Natural Disaster
Published:
Working with the American Red Cross and using the FEMA Individuals and Households Program (IHP) valid registrations data set, we explore the relationship between socially vulnerable populations and the damage that occurred to their homes from Hurricane Michael.
Recommended citation: Arnette, A., Zobel, C. W., Whitehead, M., & Luangkesorn, L. (2024). 'Analyzing Social Vulnerability as a Proxy for Damage in a Natural Disaster.' Production and Operations Management Society (POMS) Annual Conference, Minneapolis, MN.
INFORMS 2024 Panel on Artificial Intelligence and Generative AI in Analytics Practice
Published:
Panel exploring practical applications and challenges of generative AI in enterprise analytics practice.
Recommended citation: Luangkesorn, L. (2024). 'Panel on Artificial Intelligence and Generative AI in Analytics Practice.' INFORMS 2024 Annual Conference, Seattle, WA.
INFORMS Analytics 2025 Where Should Analysts Live: Organizing Analytics Within the Enterprise
Published:
This talk focuses on common management structures—centralized, decentralized, and matrix organizations—and how they impact the success of analytics projects.
Recommended citation: Luangkesorn, L. (2025). 'Where should analysts live: Organizing analytics within the enterprise.' INFORMS-Pittsburgh & INFORMS Analytics+.
PyCon USA Does Generative AI Know Statistics?
Published:
This talk discusses some experiences using Generative AI as an aid in applied analytics and walks through an example that illustrates working around its weaknesses and taking advantage of its capabilities.
Recommended citation: Luangkesorn, L. (2025). 'Does Generative AI know statistics?' PyCon USA, Pittsburgh, PA.
INFORMS 2025 Panel on AI and Optimization for Smarter Healthcare Systems
Published:
Panel on the integration of artificial intelligence and mathematical optimization in modern healthcare systems.
Recommended citation: Luangkesorn, L. (2025). 'Panel on AI and Optimization for Smarter Healthcare Systems.' INFORMS 2025 Annual Conference, Atlanta, GA.
INFORMS-Pittsburgh Panel: Mentoring in Analytics
Published:
Panel discussion on mentoring strategies and professional development in analytics.
Recommended citation: Luangkesorn, L. (2025). 'Mentoring in Analytics.' INFORMS-Pittsburgh, Pittsburgh, PA.
Fireside Chat: Exploring Generative AI in Healthcare and Emergency Response (CMU)
Published:
Fireside chat with Heinz College at CMU Healthcare Analytics and AI Management students.
Recommended citation: Luangkesorn, L. (2026). 'Exploring Generative AI in Healthcare and Emergency Response.' Fireside chat at Heinz College of Information Systems and International Affairs, Carnegie Mellon University</i>.
INFORMS Analytics 2026 Workshop Using Generative AI in Analytics: Demonstration, Pitfalls, and Practices
Published:
Demonstration and discussion of generative AI workflows, common pitfalls, and best practices presented for INFORMS.
Recommended citation: Luangkesorn, L. (2026). 'Using Generative AI in analytics: demonstration, pitfalls, and practices.' INFORMS Webinar & INFORMS Analytics+ 2026.
Download Paper
Development and Deployment of Advance Operational Planning for Disaster Response for the American Red Cross (Columbia)
Published:
Guest lecture on disaster response operational planning in Operations Research for Public Policy at Columbia University IEOR.
Recommended citation: Luangkesorn, L. (2026). 'Development and deployment of advance operational planning for disaster response for the American Red Cross.' Guest lecture in Operations Research for Public Policy (Dr. Eric Stratman), Columbia University IEOR.
Where Should the Analysts Live: Organizing Analytics Within the Enterprise (University of Alberta)
Published:
This talk focuses on common management structures—centralized, decentralized, and matrix organizations—and how they impact the success of analytics projects.
Recommended citation: Luangkesorn, L. (2026). 'Where should the analysts live: Organizing analytics within the enterprise.' Guest Lecture in Healthcare Analytics (Dr. Saied Samiedaluie), Alberta School of Business, University of Alberta.
INFORMS 2026 Doctoral Student Colloquium Industry Career Paths Panel
Published:
Doctoral Student Colloquium Industry Career Paths Panel at the INFORMS 2026 Annual Conference.
Recommended citation: Luangkesorn, L. (2026). 'Industry Career Paths Panel.' INFORMS 2026 Annual Conference Doctoral Student Colloquium, San Francisco, CA.
INFORMS 2026 Panel: AI in the ORMS Workforce: Threat, Tool, or Career Accelerator?
Published:
Panel discussion on the role of AI in the OR/MS workforce at the INFORMS 2026 Annual Conference.
Recommended citation: Luangkesorn, L. (2026). 'Panel: AI in the ORMS Workforce: Threat, Tool, or Career Accelerator?' INFORMS 2026 Annual Conference, San Francisco, CA.
teaching
Teaching experience 1
Undergraduate course, University 1, Department, 2014
This is a description of a teaching experience. You can use markdown like any other post.
Heading 1
Heading 2
Heading 3
Teaching experience 2
Workshop, University 1, Department, 2015
This is a description of a teaching experience. You can use markdown like any other post.