In order to show how technological innovation and scientific innovation are linked in the course of research in human science, I present an account of a series of innovations made in our laboratory (Distal Glove – Tactos system – Intertact server – Dialtact module). We will see how research on the technical constitution of cognitive and perceptual activities can be associated with a process of innovation. The technical devices present at each stage carry an interpretative framework that prepares the following stages. Devices which were initially developed for the purposes of performing experiments contributed both to scientific inventions and to developments with a practical and social finality.
Para mostrar cómo la innovación tecnológica y la innovación científica están vinculadas en el proceso de investigación en la ciencia humana, presento una aproximación de una serie de innovaciones realizadas en nuestro laboratorio (Distal Glove-Tactos system-Intertact server-Dialtact module). Veremos cómo la investigación sobre la constitución técnica de las actividades perceptivas y cognitivas puede asociarse con un proceso de innovación. Los dispositivos técnicos presentes en cada estadio comportan un marco interpretativo que prepara los siguientes estadios. Los dispositivos que fueron inicialmente desarrollados con propósitos de realización de experimentos contribuyeron tanto a invenciones científicas como a desarrollos con una finalidad práctica y social.
When human and social sciences (economics, management, sociology, cognitive science…) seek to understand the processes of technological innovation, they usually remain external to the object of their study, in the sense that they do not themselves contribute to the conception of these innovations (Callon
Moreover, within scientific research itself, there is invention by individuals and teams of scientists, as well as the diffusion of these inventions as “innovations” in the relevant research communities. But is it just to remember that knowledge is a human historical creation that we speak about scientific “innovation” rather than “discovery” (to find a thing or a truth that already existed)? We want to show here the possibility of a relation between technological innovations and innovation in social and human science, which would not be a purely external relation.
The aim of this article is to show the possibility of an approach where technological innovation and innovation in human science are directly associated, i.e. an approach where the products of scientific research participate in technological conception, and at the same time technological innovations participate in scientific discovery.
To do this, I propose to recount a process of innovation which occurred in our own laboratory (Distal Glove – Tactos system – Intertact server – Dialtact module). This story, subjective and somewhat simplified, will aim in particular at showing how the scientific, technological and social dimensions of the invention were entangled. I will nevertheless attempt to systematically distinguish these different conditions and motives, in order to examine their relations. It will be necessary to be attentive to the way in which, at each stage, the conditions of the scientific and technical production depend on innovations previously adopted in the team. We will see in particular how the technical environment, with the experimental devices, is carrying interpretive schemes which lead to the following innovations. I will then discuss the more general principles which can be inferred from these examples in order to better understand the processes of innovation. Before starting, however, I will give some indications as to the theoretical and methodological context of this research.
In spite of many calls for interdisciplinarity on the part of political and economic institutions, the actual relations between human science and technological developments often amount either to indifference (if not outright opposition); or else to reciprocal instrumentalisation (Mitcham 1994; Cresswell 2011). In the first case, technologies (considered as mere means) are opposed to society and culture, the latter being considered as the sole source of meaning. In the second case the “humanities” are called upon merely to promote the social acceptance of technical devices developed elsewhere and independently (or, at best, to propose minor adjustments by taking the user into account); conversely, it is the human sciences which instrumentalize technologies by taking them as a field of study —for example, the study of processes of innovation.
In order to overcome these oppositions, our research team aims to carry out
For the domain of cognitive science which will concern us here, it is a question of understanding how technical mediations participate in our cognitive activities, in particular for perception and for social interactions (but also for scientific research and technological innovation which are themselves cognitive activities). Rather than being a matter of manipulating internal mental representations, we consider that cognitive activity unfolds in an environment which is technically equipped and which extends in the space of external inscriptions. This field of research is currently being developed under the title «extended mind theory» (Clark & Chalmers 1998), and is inscribed in the general paradigm of embodied and embedded cognition (Chemero 2009).
We can begin the story we want to tell here with the visit of Paul Bach-y-Rita to our laboratory in 1995. By presenting us his sensory substitution device, the TVSS (Tactile Vision Substitution System), he brought both a highly original technical device and an opportunity for fundamental research on perception and its technical mediation.
At the technical level, the TVSS was designed for blind persons. The simple idea is that of replacing a deficient sensory modality with another one that is still available. Created in the late 1960’s by Paul Bach-y-Rita, this device converts visual information captured by a video-camera into tactile signals in the form of a 20 × 20 matrix of points. In this way the image, reduced to 400 pixels (black or white without any intermediate levels of grey) is projected onto the skin by means of electromagnetic (or piezo-electric) point vibrators, or directly by electrical stimuli (Figure 1).
For Bach-y-Rita, the origin of his idea lay at the crossroads between social preoccupations (restoring a form of vision for blind persons) and scientific interests (demonstrating the importance of brain plasticity). For us, observations on the usage of this technological innovation promised some crucial empirical elements for the debates in cognitive science in which we were engaged. Following the increasing criticism of computo-representationalism (Winograd & Flores 1986), we wished to contribute to the development of an alternative, ecological and enactive approach to cognition according to which perception was an embodied activity and not simply the reception of information (Gibson 1986; Varela
The observations on the usage and learning of the TVSS are particularly interesting for understanding spatial perception and the processes of appropriation of a new tool. If learning occurs while the camera is placed on a table and remains immobile, the perceptual capacities remain very limited and amount merely to patterns of tactile stimulation that are felt on the skin. However, if the blind person is allowed to take hold of the camera and to actively explore simple scenes, then a significant change occurs: progressively (after something like fifteen hours of learning), the subject becomes able to recognize highly complex forms, and even faces. Above all, and quite dramatically, this increased capacity to recognize shapes seems to be accompanied by an
How can perceptual learning explain the passage from the perception of stimuli on the skin, to the perception of an object “out there” in space? How can one explain the appropriation by which the technical device, which was initially perceived as an object (camera, tactile stimulator), “disappears” and becomes transparent with respect to the objects that it makes it possible to perceive? How can we explain this spectacular brain plasticity? The tactile sensory input has nothing in common with that of the retinal system, and neither does the manual control of the camera have anything in common with the control of the ocular muscles. Nevertheless the brain turns out to be able to re-organize itself to produce a perceptual world specifically characterized by the recognition and indeed the localization of shapes and objects. This involves understanding the functional relationships between the technically equipped organism and its environment, i.e. a structure of sensori-motor coupling that is sufficiently stable to induce this functional reorganization of the central nervous system (Obayashi
There are here some empirical elements in favour of a conception of the space of “objective exteriority” as
At the technical level, following a classical tendency towards discreet miniaturization, a new version of the TVSS has been developed to distribute the sensory data in the form of electrical stimuli on the tongue (TDU, Tongue Display Unit) (Bach-y-Rita 2004). For us, starting from a scientific perspective, our first idea was to create a
In order to identify the necessary conditions for the appearance of a space in which stable objects are perceived, one approach consists of deliberately
At each point in time the blind (or blindfolded) subject receives only a minimal amount of information, one bit corresponding to the presence or absence of the tactile stimulus. It has been possible to show that even in these drastically reduced conditions, the subject is still able to localize a target placed in different directions and at various distances (Lenay 1997; Siegle et Warren 2010). It is quite easy to understand that the subject should in principle be able to solve the task by means of triangulation, by aiming at the target with different positions of the arm and angles of the wrist. The spatial perception corresponds to a
The distal glove, as an initial technological innovation, opened up the possibility of an experimental methodology for fundamental scientific research on the constitution of experience in the case of tools grasped in the hand. The technical mediation made it possible to control and to vary the repertoires of action and sensation, in order to study the types of sensori-motor invariants arising from each type of mediation.
This initial technical device could inspire all sorts of new developments (Auvray & Myin 2009), for example increasing the number of captors (at the end of each finger in order to actively explore the distal space, or in a circle around a hat to give access to omni-directional perception) (Kerdegari
At the technical level, the aim was to simplify still further the possibilities for action, in order to study the perception of shapes in a two-dimensional space. The “Tactos” system makes it possible to « touch » digital shapes present on the screen of a computer (Hanneton
On the scientific level, it has been shown that the users (blind persons or blindfolded adults) can learn to localize and recognize simple shapes (Ziat
With this experimental setup, we were able to explore the way in which the perceptual activity is transformed when the technical mediation is progressively complexified, for example by studying the effects of progressively increasing the number of receptor fields (with a corresponding increase in the number of tactile stimulators). This allowed for an economy of movement and memory, and thus for perceptions which were more rapid and more precise (Sribunruangrit
At the social level, the Tactos system could be directly useful. Thus, although screen-reader technologies are now providing access to many digital activities for the blind and visually impaired persons, these solutions have the drawback of poorly conveying spatial information such as data charts and tables, drawings, graphical interfaces and web pages layouts. With the Tactos System, the users can perceive the shapes and the layout of objects on the screen. With the user engagement in learning, more complex content like maps or webpages layout become meaningful (Gapenne
Several of the first generation of blind users have become our collaborators, contributing to a new cycle of technological innovation according to the uses they imagined: bi-modality (acoustic information distributed according to the position of the receptor field) (Ammar
Once we had created a way of “touching” digital shapes on the screen, it became apparent that this digital space could be shared between several users. For this we have developed the “Intertact” server that distributes interactive and multiuser applications to Tactos users. The receptor fields of each participant are associated with a “body-image”, i.e. a shape that can be perceived by other users. When the receptor fields of one participant touch the body-image of a partner, the receptor fields of this second participant touch the body-image of the first (Figure 4). This is the situation of “perceptual crossing”, just as when looks meet each other or when there is inter-individual contact by touch. One can thus construct all sorts of spaces for games and exploration, in which the users can encounter each other, follow each other and jointly perceive objects belonging to a common environment.
On the scientific level, the technical device enabled us to construct a minimalist form of perceptual crossing in a shared one-dimensional space of action (left-right movements of a receptor field) and an all-or-nothing tactile stimulus when the receptor field encounters an object, be it the body-image of the partner, or fixed or mobile lures.
Without entering here into details, the analysis of the way in which participants succeed in the task of recognizing the other subject (clicking on the body-image of the partner) turned out to be surprising. We were lead to conclude that the recognition of another subject had to be collective before it could become individual: the perceptual activities of the two subjects had to “capture” each other, and it was only afterwards that the individual subjects could recognize each other. The feeling of the “presence” of another subject is due to this sort of mutual engagement. Thus a technical innovation led to a theoretical innovation for understanding the phenomena of social cognition. The same experimental paradigm has subsequently allowed us to propose new approaches to phenomena such as the mimicry of facial expressions or joint attention (Lenay & Stewart 2012; Deschamps 2016).
These scientific questions connect directly with the important issue of the social success of an innovation. Thus, in spite of the effectiveness of sensory substitution devices such as the TVSS, which is widely recognized, it is surprising—and disappointing—to note that they have been pretty much a failure, both socially and economically. Why is it that these techniques, which were invented in the 1960’s and experimentally validated in the 1970’s, have had so little impact on the daily life of blind and visually handicapped persons? There are many possible answers to this question: a practical effectiveness which is insufficient (none of these devices enables a blind person to drive a car); the unpleasant feeling of looking like a weird cyborg; but perhaps, above all, the lack of a quality of lived experience. One can try showing a person who has been blind from birth the image of his wife, or his own image in a mirror; one can try showing blind students sexy pin-ups; in all cases, the disappointment is flagrant. There is indeed the constitution of an object, the cognitive capacity for discrimination and categorization; but there is no
In the process of innovation described here, the work was carried out on the basis of problems that were initially of interest from the scientific point of view before taking into account social uses. And what is remarkable here is that the partial failure of the adoption of these innovations by blind people becomes a new and important scientific problem. The technological situation offers the opportunity of conducting empirical research concerning the anthropological question of the genesis of the values (emotional, ethical, aesthetic) that are attached to percepts. Our approach consists of working on the
Indeed, at the sociological and economic levels, a tactile Internet ought to make it possible to observe the formation of communities of users. The Intertact server thus offers a space made up of multiple virtual rooms that the users can explore together, “rooms” devoted either to a programme for learning simple shapes, or to educational contents (geometry, geography) (Sribunruangrit
On the technical level, making a portable version of our devices required above all that we sufficiently miniaturize the system of tactile stimulation (a 4 × 4 matrix of piezoelectric stimulators) to achieve an energetically autonomous module that could be mounted on the back of a Smartphone. One of the prime advantages of this device is that it can be used with just one hand: the thumb on the screen controls the movements of the receptor fields, whereas the index finger is placed on the matrix of tactile stimulators on the back of the Smartphone. It also allows for the exploration and perception of the graphic interface on the screen. But above all, the software Dialtact allows for rich inter-individual tactile interactions, each subject directly controlling the activation of the tactile stimulators of their partner (Figure 5).
At the scientific level, the observations made with the Intertact server have shown that a very simple design of interaction allows for the creation of a dynamics of perceptual crossing which conveys the feeling of the “presence” of another subject. Now, the Dialtact device makes it possible to envisage the study of the development of a form of tactile language which could be differentiated between different communities of users. We are currently making some initial observations on groups of alpha-test highschool children.
At the social level, the aim of this device (and the financing which helps us to develop it) is to propose applications which may interest the general population, and not merely communities of blind users. Indeed, it is a question of exploring a new market, that of tactile interfaces for access to information in a discreet manner (without having to bend one’s head to look at a screen, and without having to wear head-phones) for guidance and intimate communication.
In order to understand the process of innovation presented here, we may take up the approach proposed by Daft and Weick (1984) concerning innovation by organizations. Thus, our team of research in human science functions at each moment of the process like an “interpretative system” with regard to its scientific and social environment. At the scientific level, this environment is that of debates within communities of scientists concerning perception, social cognition and the study of technologies; at the social level, it is a question of associations of blind persons with their practices of digital technologies. The “interpretation” consists of giving meaning to events coming from these environments, in order to transform their knowledge and to decide what can be done. This interpretative activity of the organization can be seen through the actions collectively taken. Daft and Weick distinguish various types of interpretative practice: either by considering the environment as already given, in which case it is merely a case of knowing it better in order to improve the technical devices; or by considering that the environment is itself defined by interpretative practices. In this latter case the interpretative activity consists of transforming the environment by introducing new technologies and practices, and then observing their success or failure. Rather than responding to demands which are supposed to pre-exist, it is a question of inventing new uses and testing the possibility of new markets. In the latter case, the authors speak of “organisation enacting”. As noted by Brown and Duguid (1991), it is not a question of responding to empirical observations, but rather of constructing a conceptual framework and imposing it on the environment.
We may add that this interpretative framework is not only conceptual, but that it is inscribed in technically equipped practices. It is the technical devices that are currently present and used which are bearers of an interpretative framework, since they transform our perception of what is possible. Invention is not simply a choice in a situation of limited rationality (Newell et al. 1959); it is rather that the field of possible choices, of solutions that can actually be achieved at a given moment, is itself constituted by the technical devices that one has available. Thus, creative imagination is supported by technical devices. Amongst the four stages of the “inventive process” identified by Usher (1929) (1-« recognition of a new or incompletely gratified want » ; 2-« setting the stage » ; 3-« act of insight » ; 4-« critical revision ») it seems to us that the second is the most important. Once the technical elements (together with the know-how concerning their usage) are assembled, the “invention” appears as a solution that is present in a field of accessible possibilities, without any need of a mysterious “insight”.
The technical constitutivity of human activities necessarily implies a form of “advance” of the technical conditions with respect to what they will reveal when they are actually deployed. At each stage, we well understand how the Distal glove gives rise to the possibility of Tactos for the perception of bi-dimensional shapes; how Tactos gives rise to the possibility of Intertact for interactions on the Net; and how Intertact gives rise to the possibility of Dialtact for mobile tactile communication. It is equally evident that when we first developed the initial minimalist devices towards the end of the 1990’s, we didn’t have the least idea of these perspectives that were subsequently discovered. Insofar as the technology is constitutive, conditions and problems that govern the development of an innovation are always different from the issues and conditions for success that arise through its use. Knowledge of the market, of the users, and their needs always comes after the event, since at each stage of the development of an innovative technological device, the device itself opens up unsuspected possibilities and problems which did not even exist at the moment of its inception.
We have seen that these experimental devices were an important source of innovation, whose consequences were not only scientific but also practical and social. And when it is a question of proposing practical uses for these devices, a second cycle of development is put in place, this time in close relation with potential users who can also themselves contribute to the conception of new functionalities.
We have seen in the cases presented here that the innovations correspond first and foremost to devices created for the purpose of fundamental research. It is only afterwards that the practical deployment of these devices in their experimental context rendered visible some possible new innovations with social value. There is a link between fundamental research and technological development, but it is not a relation of “application”. For example, it is not knowledge supposedly acquired concerning “natural” cognition which guided the conception of cognitive technologies such as writing, nor is it knowledge already acquired on perception that would have led to the invention of perceptual supplementation systems. Technological innovation does not result from a process of explicit reasoning through the application of knowledge which already exists, because on the contrary it is the technical device which participates in the constitution of the knowledge in question. The theory of the technical constitutivity of cognitive activities is prolonged by a technical constitution of this theory itself. This is a basic characteristic of an experimental method which, beyond the simple verification (or refutation) of prior hypotheses (the hypothetico-deductive method), expects from the situation created by actually putting the experimental device into practice, that it should reveal phenomena which otherwise could not even have been imagined. However, whereas technological innovations do ceaselessly open up unexpected possibilities, they also allow us to better understand what is happening. Understanding how variations in technical devices modify cognitive or perceptual activities, amounts to better understanding these activities themselves (spatial localization, shape recognition, recognition of another subject, joint attention…). Our research aims to understand the technologies to better understand the human, i.e. an unfinished being, always engaged in a historical and technological becoming. This is what solves here the paradox of innovation that can simultaneously be scientific discovery.
The author would like to thank the Hauts-de-France Region and the European Regional Development Fund (ERDF) 2014/2020 for the funding of this work.
Charles Lenay is Professor of philosophy and cognitive Science. His research interests concern cognitive technologies: how tools and technical devices participate in cognitive activity in the broad sense –perceiving, reasoning, memorizing, interacting. For this he created a minimalist experimental paradigm (Distal glove, Tactos, Perceptual crossing) allowing a dialogue between cognitive psychology and phenomenology. As Director of the Research Unit COSTECH (Knowledge, Organisation and Technical Systems) at UTC (Université de Technologie de Compiègne / Sorbonne Université) he has directed 11 PhD, published nearly 200 articles, books and lectures, and filed 3 patents. In spring 2017, he was appointed to the rank of Knight in the French Order of Academic Palms.
Address: Research Unit EA2223 COSTECH, Human Sciences and Technology Department. UNIVERSITE DE TECHNOLOGIE DE COMPIEGNE, Rue du docteur Schweitzer, CS 60319,60203 COMPIEGNE CEDEX, FRANCE. Email: charles.lenay@utc.fr. http://www.costech.utc.fr/spip.php?auteur4
Equipe EA2223 COSTECH (Knowledge, Organisation and Technical Systems) and Master User eXperience Design at Sorbonne Université / Université de Technologie de Compiègne.