🦙 Suri: Multi-constraint Instruction Following for Long-form Text Generation

Chau Minh Pham, Simeng Sun*, Mohit Iyyer
University of Massachusetts Amherst
*Now at NVIDIA
[Paper] [Code] [Dataset] [I-ORPO Model] [SFT Model] [ ]

TLDR;


1. We introduce a new dataset, Suri, which contains multi-constraint instructions (~10 constraints per instruction) and lengthy gold responses (between 2k and 5k words).

2. We propose I-ORPO, a variant of ORPO (Hong et al., 2024) that handles chosen and rejected prompts rather than chosen and rejected responses.

$$ \begin{align} \mathcal{L}_{\text{I-ORPO}} = \mathbb{E}_{(x_w, x_l, y)} \left[ \mathcal{L}_{\text{SFT}} + \lambda \cdot \mathcal{L}_{\text{I-OR}} \right] \end{align} $$

where

$$ \begin{align} \mathcal{L}_{\text{I-OR}} = - \log \sigma \left(\log \frac{\textbf{odds}_{\theta}(y|x_w)}{\textbf{odds}_{\theta}(y|x_l)} \right) \\ \textbf{odds}_{\theta}(y|x) = \frac{P_{\theta}(y|x)}{1- P_{\theta}(y|x)} \end{align} $$

3. We find that fine-tuning Mistral-7B-Instruct with I-ORPO and SFT results in significantly longer generations and decent constraint-following ability. In addition, SURI-I-ORPO generations are generally preferred for their coherent and informative incorporation of the given constraints.

4. We observe that GPT-4o does not reliably evaluate the constraint-following ability of long-form text generations.

Suri Dataset


Access the full dataset here: Suri. To also access the Books3 subset, please refer to our Github repository for detailed instructions.

Click on the links below to view responses to the sample instruction. Note that you need to click on each button to hide the text.
[Gold Response] [Mistral-7b-Instruct] [I-ORPO] [SFT]

Instruction

Compose a detailed and engaging lecture on the development of architecture from the late Middle Ages through the early Tudor period, focusing on how socioeconomic changes influenced architectural styles and practices, the rise of a new consumer class, and the evolution of residential, religious, and commercial buildings in England. - Begin the lecture with a recap of previous discussions, emphasizing the impact of economic crisis and demographic changes on architecture, and express an intention to balance previous focuses by shifting attention to the living conditions of ordinary people during the period. - Introduce the Black Death and agricultural recession as pivotal events that led to significant changes in the countryside, including depopulation and its surprising benefits for the survivors in terms of improved living standards and economic opportunities. - Detail the reversal of land management trends among landlords around 1380-1410, highlighting the shift towards renting lands for a guaranteed income and the emergence of a new class of tenant farmers who significantly influenced social and architectural landscapes. - Describe the architectural characteristics of timber-framed farmhouses built by this new class, noting regional variations and specific examples that have survived to the present day. - Explain how these socioeconomic changes extended to affect the design and function of the houses of the rich, emphasizing decreased household mobility, the need for more storage, and the consequent rise in local consumption of goods. - Discuss the construction of towers and gatehouses by the aristocracy as symbols of status, including specific examples and how these structures evolved to become more integrated with residential spaces. - Address the development of commercial architecture, particularly inns, taverns, and alehouses, and how these establishments catered to different aspects of social life and commerce, including the architectural distinctions among them. - Transition to church architecture, noting how royal patronage from the late 15th to early 16th centuries led to significant ecclesiastical constructions, and describe the stylistic changes and specific examples of royal commissions. - Discuss the impact of economic conditions on monastic architecture, detailing how the quest for individual comfort and privacy among monks and abbots led to architectural innovations within monastic communities. - Introduce brick as a significant architectural material in the 15th century, explaining its advantages, the introduction of manufacturing improvements from northern Europe, and its impact on the aesthetics and construction of buildings. - Conclude with a reflection on the broader social and cultural shifts of the 15th century, including the rise of a more mobile and less stratified society, and hint at the dramatic changes to come with the Reformation, setting the stage for future lectures. - Throughout the lecture, maintain an engaging and informative tone, incorporating anecdotes and specific details to illustrate points, and encourage listeners to explore these architectural examples firsthand.

Gold Response

How the Middle Ages were Built: Coming of Age, 1408-1530 In my last lecture we took a look at a century that was dominated by economic crisis and demographic collapse. I described how out of this turmoil came a distinctive architectural style often described as the perpendicular style. In doing so we mainly concentrated on polite architecture, on castles, churches and palaces. I want to start tonight by righting that balance as we have not heard, for a while, about where more ordinary people lived. The Black Death and agricultural recession had a huge impact on the countryside. Between 1370 and 1520 as many as 2,000 villages were abandoned and many more shrank in size leading to the abandonment of half a million buildings. Yet this drastic depopulation of the countryside was, surprisingly, largely to the advantage of the people who lived there. Because there were fewer people basic foodstuffs were cheaper and more meat and fish was available: as a result people’s diet markedly improved. Wages were also higher, unemployment was lower, the choice of jobs was wider and women took on skilled jobs like weaving. But the real change was the opportunity presented by landowners. Three lectures ago I described how between 1184 and 1215 landlords had moved to take their own lands into direct management but between 1380 and 1410 this process was reversed. Landlords realised that agriculture was now unpredictable and brought low profits and that in contrast renting their land would bring a guaranteed income. This change had a profound effect from around 1400 bringing a new and powerful group of farmers to prominence. A farmer is technically-speaking someone who pays a fixed rent or farm for their land and this new group of tenants acquired perhaps as much as a quarter of all agricultural land in England. Some were former employees of the lords, many were the richer peasants. This was a redistribution of wealth away from the aristocracy to a new class of people who created new social relationships and methods of production. Across large parts of England these men built themselves houses. Hundreds of timber framed farmhouses from the period 1430 to 1530 survive today in the richer parts of the country, in Kent and Sussex as far west as Hampshire and north into Essex and southern East Anglia. They share a broadly similar plan with a central hall open to the roof with two-story blocks at each end containing chambers. The whole was contained under a single roof and in some parts the upper floors were jettied out over the lower ones. Watermill House, Benenden, Kent, for instance, is still much as its late fifteenth century builder left it. In the midlands and western England similar types of building are found, although generally of a slightly humbler nature. What is really important about these houses is that they were detached from their farm buildings in such a way that there was nothing inherent in their design that revealed that they were lived in by farmers. These families had their own halls, just like their social and economic superiors, they had their separate bedroom, parlour and kitchen; some even had a screens passage. Beams were carved, doors panelled and windows filled with glass. Decorative metalwork, drains and gardrobes were all now commonplace. The owners of these buildings were active consumers of a wide range of household goods. Decorative ceramics, in particular, would have been ubiquitous, but furnishing textiles and furniture were also in great demand. To the English countryside was now added a type of housing below that of the gentry and above that of the peasant house, well built and tuned to the lives and aspirations of a new consumer class. The same processes that created the farmers and their farmhouses had an impact on the lives and the houses of the rich. The most important development for them from around 1300 was a noticeable decrease in household mobility. Early medieval households were always on the move but during the fourteenth century noble households moved about less, some stayed put for most of the year. This had no single cause, but the fact that so much land was now tenanted to farmers meant that lords could not live off their own estates as they travelled around and at the same time had less reason to visit their far flung holdings. Much longer periods spent in one place affected the design and economy of the great houses. For instance much more storage space was needed for household goods and provisions, for wine in cellars and for textiles in wardrobes. Permanent staff had to be accommodated in ranges of lodgings and these people had to be provisioned all the year round. Great houses became huge local consumers of firewood, candles and foodstuffs stimulating local markets in a way they had not done before. Thus, as I described in my last lecture, great houses became larger, more complex, and more richly appointed and most lords chose one, or at most two houses that could be brought up to the expected standards. Most landlords saw their income reduce during the fifteenth century but the long period of decline slowed down in around 1470 and from then until the 1520s most saw an increase in revenue. This led to an upswing in aristocratic building. Towers, either free-standing or as gatehouses, had been at the core of English aristocratic building from late Saxon times and they never became unfashionable, they were a dominating presence, expensive to build, conferring status on a place as well as being agreeable places to live with great views. During the late fourteenth century a small group of aristocrats built a new sort of tower which was, in effect, a complete residence integrating hall, chapel, lodgings and domestic offices. The best of these were at Warkworth Castle, Northumberland, dating from 1377 and Old Wardour Castle, Wiltshire started in 1393. These were precursors to the huge integrated tower started by Henry V and designed by the King’s mason, Stephen Lote at Shene (modern Richmond upon Thames) in 1414. It was almost certainly this spectacular new royal palace at Shene, nearing completion in the 1430s, which re-energised the constructionof great towers in a number of new aristocratic mansions. The men who built these were close to the royal family: Ralph, Lord Cromwell, Treasurer of England, built Tattershall in Lincolnshire in 1434-46; the most powerful man in Wales, William Herbert, earl of Pembroke built at Raglan Castle, Gwent from 1461 and William Lord, Hastings Edward IV’s Chamberlain constructed a tower at Ashby de la Zouche in Leicestershire in 1474-83. Lesser men too followed with towers at their houses such as Lord Lovell at Minster Lovell and Sir Robert Harcourt at Stanton Harcourt, both in Oxfordshire. By the end of the fifteenth century these towers became less independent structures and more integrated with the principal lodgings of the house. Two buildings are particularly in this process. The first is Edward IV’s new lodgings at Nottingham Castle. There the king commissioned a massive new tower integrated with a fine range of lodgings. The lodgings were built against the outer curtain wall but they faced inwards with a series of seven two-story bay windows to the inner court. The king’s private rooms were positioned in the tower, but approached by three magnificent chambers of state. This was a radically new and important building. Here you see for the first time the use of bay windows as a decorative feature, a feature that was to go on and define Tudor architecture for a century or more. Then at Greenwich, in about 1500, Henry VII’s master mason Robert Vertue took this to the next stage. The royal palace there was not fortified in any way; it presented a long range to the river Thames with bay windows looking out both ways. In the middle, as part of the main range, a tower rose containing the king’s privy rooms. Greenwich was to become a blue print for houses of status and pretension for the following twenty years. In the autumn when I come on to speak of the great houses of the Tudor period, it is back to Greenwich that I will refer. This was a seminal building in the history of English domestic architecture. But the riverfront at Greenwich was only one range of a palace with two large courtyards. For houses of the first order a single courtyard would not suffice. Most large castles were planned on the basis of two courtyards, wards or baileys; an outer one for the household, retainers and services and an inner one for the lord’s family. In the late fourteenth century the largest courtyard houses adopted a second courtyard (or even a third) for exactly the same reason. Thus a very large country house like Dartington Hall or a big urban palace like the Archbishop of Canterbury’s at Lambeth, London would have two courtyards. The most impressive survival of this type is Wingfield Manor, Derbyshire (1440-1456) which can be seen as the precursor to the next generation of top-end aristocratic houses, the Duke of Buckingham’s Thornbury Castle, Bristol (c.1510-21), and Thomas, Cardinal Wolsey’s Hampton Court (1514-28). In most of these double courtyard houses the great hall was sited in the range between the two courts acting as the vestibule to the more private inner courtyard and the lord’s own lodgings beyond it. Now changing lifestyles of the aristocracy had wide ranging impacts. One of these was that as they travelled they now no longer stayed at their own houses – having now so few and relied much more on renting accommodation for the night. From early times people had rented out space in their houses for travellers, but by the thirteenth century Inns became common in most towns of any size. Inns were like hotels. Taverns, which provided good quality food and drink, including wine, were more like restaurants. Alehouses were pubs providing basic food, ale and on occasion rooms. From the sixteenth century these distinctions were recognised in law and defined licensing restrictions and the obligations of landlords. From a census taken in 1577 we can calculate that there were then around 17,367 alehouses, 1,991 Inns and 401 taverns in England. Alehouses were often just people’s ordinary residences, although larger houses with cellars were more successful venues. This is because before the 1450s when more alcoholic beer, fermented with hops, became common, ale which contained only malt yeast and water, needed to be kept cool if it was to last. The larger houses too would have had attached brew houses where ale, and later beer, was made. Taverns were also essentially houses, although larger and normally with cellarage for the storage of wine. Before the sixteenth century it was only the Inn that had a distinctive architectural expression. Inns were commercial ventures constructed by landowners, often monasteries to make money. The powerfully atmospheric George Inn, Norton St. Philip, Somerset, was built in the late fourteenth century by the Carthusian Priory of Hinton in order to provide accommodation for merchants visiting the market and more generally to stimulate trade. It is typical of a type of inn that presents a fine stone façade to the street with a hall, kitchen and rooms in a single range. A narrow archway led to the rear where, perhaps, there were stables. Similar, but later (c.1480) is the pretentious George Inn Glastonbury, Somerset, with a fashionable stone façade bearing the arms of the abbot who built it and those of Edward IV. The New Inn, Gloucester, is undoubtedly the most spectacular surviving example, but of a second type. Built by the Abbey in 1450 it is a three stories timber-famed building built around a central courtyard. On the street front there are shops and a passageway between them leads to the central court surrounded by upper floor galleries. These led to over twenty rooms. These were large, 20ft by 12-15ft and would have contained several beds for up to half a dozen occupants so the whole inn could accommodate perhaps 200 when full. There was a hall and kitchen and as series of parlours for private dining. The New Inn could accommodate the travelling household of an aristocrat with ease. The most important guests would dine in their own rooms which were furnished with tables chairs and benches. The less well off dined in the hall and shared beds with other travellers and a variety of bed bugs. A bed in a cheap inn would cost a penny a night, less than the cost of dinner; the swankier establishments, such as the George at Glastonbury would be more. But let me move on from the lives and houses of the aristocracy and turn now to the church. In my last lecture I explained that church building in the fifteenth century was not generally driven by the aristocracy or churchmen. I showed you churches such as this one in Saffron Walden which were the products of merchants and squires. I also explained that most parish church building was about extension and adaptation not complete rebuilding. There was, however a big exception to these points: one that I now need to address. For between 1415 and 1530 English Kings again became amongst the greatest patrons of church architecture in Europe. Henry V planned three monasteries next to his royal palace at Shene and completed a large Charterhouse there, Henry VI founded chapels at Eton and King’s College, Cambridge; Edward IV started a chapel at Windsor and a friary at Greenwich and Henry VII commissioned a lady chapel at Westminster and a friary at Richmond; most of these works only came to completion under Henry VIII. The royal monasteries are no more, but at Windsor, Westminster and Cambridge three remarkable buildings remain. They represent between them the stylistic changes that began towards the end of the fifteenth century, a process of increased richness or busyness of ornament and decoration. For those of you who have followed my lectures this should be no surprise. For every style of architecture that I have mentioned which started plain and restrained became, in due course elaborated and decorated. This is, in fact, a characteristic of English architecture, and we see it very clearly in the great royal commissions around 1500. In 1471 Edward IV commissioned a new chapel of St. George at Windsor Castle. This was to be an even more splendid setting for the services of the Order of the Garter in which the king took a special interest, but it was also to glorify the Yorkist line and provide a mausoleum and chantry for the Edward himself. The architect was Henry Janyns who had worked at Eton College for Henry VI Many of the elements of its remarkable interior will be familiar from Gloucester Abbey choir and Canterbury Cathedral nave both of which we looked at in my previous lecture, but the combination creates quite a different effect, not at all vertical, in fact the nave seems wider than it is tall. This is because the extensive use of four-centred arches (which have arcs of very low pitch) make the ceiling more like a cove than a vault. These arches became increasingly popular from around 1400 and almost ubiquitous after 1500 creating wide openings allowing in lots of light. St. George’s was finished in the reigns of Henry VII and VIII by which time another royal commission, conceived in a spirit of rivalry with Windsor, had been started. Henry VII commissioned a new Lady Chapel for Westminster Abbey which was to triple as a shrine to his pious uncle Henry VI and a chantry for himself and his family. This was the most ostentatious addition to the abbey since Henry III and was the triumphal statement of a nouveau riche king striving to be more lavish in every way than the chapel at Windsor. So rather than blank arcading between the arcade and the clerestory as at St. George’s, and elsewhere there is a band of niches each containing wonderfully expressive and lifelike statues. In the middle of the chapel was to be Henry VII’s chantry behind a gilt-bronze screen containing a monument to the king and queen, an altar and reredos with gold-covered figures, all topped by one hundred nine-foot long burning tapers. Above this rides the fantastical fan vault with stalactite-like pendants falling from the conical fans. The weight of this is taken by external piers fashioned as octagonal turrets crowned by ogee domes. But these are not recognisable as piers as they share the same grid of tracery as a series of tall bay windows squeezed between them creating a continuous cage of tracery round the lower part of the chapel. King’s College, Cambridge, had been begun by Henry VI in 1448 but in 1499 was still less than half completed. The finishing touches were only finally installed in the 1540s under the patronage of Henry VIII. The chapel combines more brilliantly than any other building the principles of design begun at Gloucester in the 1340s and 50s. The vaults, windows and panelling are perfectly integrated into a single cage of tracery, yet the detail is not what you see, for it combines to create a truly monumental space. The walls seem to be entirely of glass and the vault seems to confound gravity. Externally it is clear how this illusion was achieved as the elevation is broken up by the rhythmic verticality of massive buttresses. But even these are concealed by a low-level chapels which hide the full depth of the buttresses at ground level. While no English monastery built anything like the crown, we should not forget that the monasteries were still amongst some of the most important patrons of architecture. The monasteries did not escape the economic crisis coomon to landlords after the Black Death. During the first half of the fifteenth century the value of land and rents fell and as a consequence many monasteries fell into debt. Some smaller houses merged or, in some cases, closed and most communities, trying to economise, shrank in size. As a result, although there were a few abbeys where significant programmes of innovative building were underway in the fifteenth century, this was not the great age of the monastery. Some abbey churches were modified: the great Cistercian churches at Rievaulx and Fountains, for instance, underwent reordering to take account of their smaller communities, to make room for burials and for nave altars. But most monastic architecture was now driven by a gradual dilution of community life in favour of individual privacy and comfort. This trend was given impetus by the Cistercian Pope, Benedict XII, who authorised half the brothers at a time to eat meat away from the communal refectory. This permanently undermined the sense of community and encouraged individualism and independence. In 1421 Henry V accused senior Benedictines of a life of complacency, luxury and worldliness. To a degree he was right, the rules, once so strictly observed had been weakened and bent and although few monasteries were vice-ridden many were complacent and worldly. Monks had personal property, guest houses and holiday homes and most abbots’ lodgings are rebuilt giving them more privacy and luxury. Possibly the most magnificent, and certainly the best surviving, was the lodging range built by prior Singer at Wenlock Priory, Shropshire in the 1490s. Here the east side of the infirmary courtyard was occupied by a colossal range faced with a grid of windows and buttresses forming the front of a long corridor that gave access to the main rooms. The Prior’s hall, with an elaborate carved wooden roof, dais, fireplace, wall paintings and other luxurious fittings was served by its own private kitchen below. Next door was his private chamber complete with its own gardrobe. It was not only the abbots whose lives became more comfortable; the lives of many individual monks were transformed too. At Durham a new kitchen of great splendour and technical virtuosity was built in 1366-70 to supply the monks with roasted meats which they ate in an agreeable dining room called the Loft where each monk had his own silver cup. The austerity of communal life was relieved by a spectacular new dormitory built in 1398-1404 with individual cubicles each lit by its own low level window while the main room was lit by larger windows above. The seats in the reredorter were partitioned off to give privacy as the monks relieved themselves and in the new cloister openings were glazed and the walks furnished with individual panelled carrals for study. If they should tire of all this Durham owned a rest-house, Finchale Priory, sited in a pretty spot by the river Wear. Here, in groups of four, the brothers of Durham could rest, enjoy themselves and go for country walks. These changes, and others, were accelerated by an improvement in economic conditions in the 1470s, and by the 1520s almost all estates had seen a noticeable rise in income. This boost led to a new wave of monastic construction, most not directed to the liturgical or economic needs of the monastery, but directed to display. Church towers were very popular as were showy gatehouses, guest lodgings and prior’s quarters. Abbot Marmaduke Hoby of Fountains Abbey constructed a colossal bell tower 160ft high with statue niches in the massive corner buttresses. He was not the only one, other northern abbeys at Shap and Furness built similar towers. The tower at Bolton Abbey, begun in 1520, bore an inscription celebrating the abbot who began it. At St. Osyth’s Priory, Essex, Abbot Vintoner built a flashy new gatehouse faced with beautifully ornamented flushwork containing richly carved niches. This was designed to be seen from without, for the other elevation, facing the conventual buildings, was much plainer. I want to move on now from these breathtaking structures that are at the crescendo of gothic architecture in England to a much more mundane issue, but one that is no less important in understanding the fifteenth century. That is brick. The Romans had used brick extensively, and their buildings were ruthlessly plundered for nearly a thousand years. But it was not until the fourteenth century that it started to be manufactured again on any scale. At first it was popular in counties where there was no ready supply of building stone, particularly Norfolk, Suffolk and Kent. It was also convenient in towns like London and Hull where the demand for building materials was particularly high. However in these cases it was normally concealed by rendering or used internally under plaster. It was often also used in vaults where the ribs were of stone and for the hearths and chimneys of fireplaces where its fire resistance was understood. In the fifteenth century brick moved from being a material which met special constructional needs to one of choice. This was mainly due to improvements in manufacture introduced from northern Europe. Makers, many of whom were Flemings, Germans or Dutch, began to produce bricks of a uniform size and colour and could do so in huge quantities relatively cheaply. These craftsmen also knew how to vary the brick colour in the firing and how to make moulded and shaped bricks for decorative effect. These possibilities re-launched brick as a high class building material capable of achieving quite complex architectural effects. In 1414 Henry V agreed to its extensive use at his new palace of Sheen and its associated monasteries and not long afterwards his courtiers followed suit. One of the significant benefits of brick was that it could be made anywhere that had good brickearth. Ideally this would be directly adjacent to the building site thus removing transport costs, one of the most expensive elements in any project. An expert overseer was necessary to identify the earth, particularly to ascertain whether there were suitable iron compounds to achieve a fashionable red colour. After that cheap labour could be used to knead the clay, mould it into bricks, dry them and stack them in either a kiln or a clamp (a pile of dried bricks under which a fire could be stoked). Kilns were permanent and could be re-used, but were smaller. A clamp, such as built at Kirby Muxloe Castle in 1483 fired 100,000 bricks, not an unusually large number. By the early sixteenth century there were hundreds of brick makers in England, some small operators, others contractors working on an industrial scale producing many millions of bricks for a building like Hampton Court palace. At first it was not only the manufactures who came from the continent, for the designers and bricklayers were foreigners too. But soon English masons became proficient in designing and building brick structures and adapted them to English ways of building. All the earliest major brick buildings are associated with the court of Henry VI and include the great Castle at Herstmonceaux, Sussex, begun in c.1441 by Henry’s Treasurer, Sir Roger Fiennes. The streams of design that these early brick buildings contain come together in Henry VI great work of piety at Eton. Eton College was one of the largest and most important religious foundations of the late middle ages established by Henry VI in 1440 to offer prayers for his dynasty and to provide education, ultimately, for 70 scholars. The vision, as it emerged, was on a staggering scale and was never completed, but enough survives of Henry VI’s work to understand what was new about it. This was a college much like William of Wykeham’s New College Oxford, although the vast scale of the chapel proclaimed its primary purpose as a royal chantry. What was new was the comprehensive approach to the use of brick as a high status decorative material including extensive diapering on the principal elevations. Also new was the careful mixture of brick and stone: stone used for the windows, arches and doorways and as a material for string courses and buttresses. A new type of window was extensively used. These were much simpler openings in square heads without any cusping. Such windows appear, at this time in a number of prominent brick buildings. So what was happening at Eton? We know that a large number of the brickmen (as they were called) were foreign and were perhaps employed to create the ambitious diaper patterns. We also know that the king impressed many native brickmen too, many of them from prominent patrons who were themselves constructing large brick houses. Eton College thus became a building in which all the latest ideas about brick construction were used and from which ideas were taken and spread to buildings elsewhere. In some senses Eton became the seminal Tudor structure, the father of a generation of buildings from Hampton Court to the Vyne, Hampshire. Henry VIII came to the throne in 1509. The country over which he ruled was a very different one to that which Henry V had known a century earlier. During the fifteenth century the sense of Englishness that had been developing since the thirteenth century was consolidated. The shift in economic influence from the aristocracy towards the merchants and the gentry created a self-conscious and assertive political community represented in parliament. The Peasants Revolt of 1381 had ensured that this community would not, as in France for instance, make the common people entirely responsible for direct taxation, giving the upper classes fiscal immunity. That all classes shared the burden of taxation meant English society was far less stratified and much more mobile than in most northern European countries. Englishmen now spoke a single language that was rapidly becoming the official language of administration and culture. The invention of printing meant that from the 1480s even relatively poor people could own a book in English and probably 30% of the population could read it. These people were independent and determined and were the people who were now building England. Architecture was no longer just for the Abbot, the Bishop, the Prince and the Aristocrat. It was for the merchant, the gentry, the farmers and the artisan. While Rome had been the inspiration for building in England from 410 until 1300, and God had been the inspiration after that, it was now the independently minded Englishman represented I think best by Robin Hood. Of course nobody knew in 1509 that in only thirty years the whole infrastructure of everyday life was to be blown away by the Reformation. This remarkable little island with a strong independent culture, populated with buildings of originality and beauty, with a flourishing religious life and a wealthy and influential middle class was to fundamentally change. So were its buildings, cut off from streams of thought on the mainland. But this is the story of my next lecture. You will have to wait until October 10th to hear about it though! In the meantime have a great summer and I hope that you will take the chance to visit some of the amazing buildings that I have discussed in the last year.

Results


Suri-I-ORPO and Suri-SFT generate substantially longer text without increasing the number of repetitions.

> We measure the number of tokens and 5-gram repetitions in 5,000 generations from the test set. The left figure shows that generations produced by our fine-tuned model are significantly longer than those produced by open-weight models (Mistral-7b-Instruct, Llama-3-8B-Instruct, Mixtral-8x7B-Instruct). The right figure demonstrates that as the generations become longer, they do not necessarily contain more 5-gram repetitions, which is a sign of quality degradation.


I-ORPO improves ranking accuracy across different level of instruction specificity.

> Each instruction specificity setting can be interpreted as the number of total constraints and the number of those included constraints that are corrupted: (M,M), (M,M/2), (M,1), (M/2,M),(1,M). For example, in the (M, M/2) setting, both instructions include all constraints, but only half of the constraints are violated. The table below shows that, across all settings, I-ORPO outperforms SFT and open-weight models.


GPT-4o does not reliably evaluate the constraint-following abilities of long-form text generations.

> We measure the agreement between GPT-4o and human annotations in assessing the number of constraints satisfied by text generation. The table below shows that GPT-4o often disagrees with human annotations.


Our human evaluation shows that, while our fine-tuned models are both generally effective at following constraints, Suri-I-ORPO is preferred over Suri-SFT for coherent and informative constraint satisfaction.

> On average, Suri-I-ORPO and Suri-SFT meet most of the included constraints, achieving satisfaction rates of 67-68% and partial satisfaction rates of 16-17%.


> Human annotators consistently prefer Suri-I-ORPO to Suri-SFT for about 60-70% of the time across all three categories:
- Informativeness: Which generation provides more details about the constraint?
- Coherence: Which generation effectively integrates the constraint with the rest of the text?
- Readability/Enjoyability: Which text sample is easier to read overall?

Citation


      @misc{pham2024topicgptpromptbasedtopicmodeling,
        title={TopicGPT: A Prompt-based Topic Modeling Framework}, 
        author={Chau Minh Pham and Alexander Hoyle and Simeng Sun and Philip Resnik and Mohit Iyyer},
        year={2024},
        eprint={2311.01449},
        archivePrefix={arXiv},
        primaryClass={cs.CL},
        url={https://arxiv.org/abs/2311.01449}, 
      }